REVIEW OF RECENT METHODOLOGICAL DEVELOPMENTS IN GROUP-RANDOMIZED TRIALS: PART 2 - ANALYSIS

Abstract

In 2004, Murray et al. published a review of methodological developments in both the design and analysis of group-randomized trials (GRTs). Over the last 13 years, there have been many developments in both areas. The goal of the current paper is to review developments in analysis, with a companion paper to focus on developments in design. As a pair, these papers update the 2004 review. This analysis paper includes developments in topics included in the earlier review, such as methods for parallel-arm GRTs, inference for conditional and marginal effects, and new topics including methods to account for multiple levels of clustering and alternative estimation methods such as augmented GEE, targeted maximum likelihood and quadratic inference functions. We also examine developments in dealing with missing outcome data, including doubly robust approaches, software available for analysis, and analysis of alternative group designs (including stepped wedge GRTs, network-randomized trials, pseudo-cluster randomized trials and individually-randomized group treatment trials). These alternative designs, like the parallel-arm GRT, require clustering to be accounted for in both their design and analysis.

INTRODUCTION

In a group-randomized trial (GRT), the unit of randomization is a group and outcome measurements are obtained on members of those groups.¹ Also called a cluster-randomized trial or community trial,^2–5 a GRT is the best comparative design available if the intervention operates at a group level, manipulates the physical or social environment, cannot be delivered to individual members of the group without substantial risk of contamination, or under other circumstances (e.g., a desire for herd immunity in studies of infectious disease).^1–5

In GRTs, outcomes on members of the same group are likely to be more similar to each other than to outcomes on members from other groups.¹ Such clustering must be accounted for in the design to avoid an under-powered study and in the analysis to avoid under-estimated standard errors and inflated type I error for the intervention effect.^1–5 For analysis, regression modeling approaches are generally preferred and most commonly used because of their ease of implementation.⁶ Several textbooks now address these and other issues.^1–5

In 2004, Murray et al.⁷ published a review of methodological developments in both the design and analysis of GRTs. In the 13 years since, there have been many developments in both areas. The goal of the current paper is to focus on developments in analytic methods, including those relevant to designs described in a companion paper that focuses on developments in GRT design.⁸ As a pair, these papers update the 2004 review. With both papers, we seek to provide a broad and comprehensive review to guide the reader to seek out appropriate materials for their own circumstances.

DEVELOPMENTS IN THE ANALYSIS OF PARALLEL GROUP-RANDOMIZED TRIALS

Methods for Superiority, Equivalence, and Non-Inferiority

In GRTs, superiority trials are more common than equivalence or non-inferiority trials: a PubMed search by one of the authors (DMM) of studies published in 2015 identified 562 superiority GRTs but only 1 equivalence GRT and 2 non-inferiority GRTs. Similarly, developments in the methods literature have focused on superiority GRTs, with developments for equivalence and non-inferiority GRTs limited to small sections in two of the more recent textbooks^2,5 and a review paper on sample size methods.⁹ As a consequence, the current review paper focuses on superiority GRTs.

Methods for Intention-To-Treat and Alternative Intervention Effects

In GRTs, protocol violations can lead to non-compliance at either the group- or member-level.⁵ In order to minimize bias, intention-to-treat (ITT) principles are recommended at both levels rather than “on-treatment” and “per-protocol” analyses.^2,4,5 While group-level protocol violations are usually easy to identify, member-level compliance may be more difficult to ascertain in practice.² Jo et al. demonstrate that analyses which ignore compliance information could be underpowered to detect an ITT effect and propose a multilevel model combined with a mixture model.¹⁰ Implications of group-level non-compliance can be considerable in GRTs, given the small number of groups that are randomized in many GRTs.

Methods Based on the Randomization Scheme

Matching or stratification in the design has been recommended for some time as a way to ensure baseline balance on important potential confounders,¹ with constrained randomization more recently developed.¹¹ Recent reports suggest that most GRTs follow this advice.^12–15 Matching and stratification in the design can be ignored in the analysis of intervention effects, without harm to the type I error rate, and often the saved degrees of freedom will improve power.^16,17 Recently, Donner et al. reported that ignoring matching can adversely affect other analyses, such as analyses that examine the relationship between a risk factor and an outcome;¹⁸ for this reason, investigators considering pair-matching should consider small strata instead (e.g., strata of 4). Li et al.¹⁹ compared model-based and permutation methods in the context of constrained randomization adjusting for group-level covariates. They found that both the adjusted F-test and permutation test maintained the nominal size and had improved power under constrained randomization compared to simple randomization.

Model-Based Methods

Model-based methods can be broadly classified according to the interpretation of the model parameters. Conditional model parameters are typically estimated using mixed-effects regression via maximum likelihood estimation (MLE) and are referred to as cluster-specific effects (or as subject-specific effects in the longitudinal analysis literature). Effects are conditional on the random effects used to account for clustering and on other covariates included in the analysis. Conditional models are often recommended for studies focused on change within members or on mediation analyses.⁷ Parameters of marginal models are usually estimated using generalized estimating equations (GEE).^20,21 They define the marginal expectation of the dependent variable as a function of the independent variables and assume that the variance is a function of the mean; they separately specify a working correlation structure for observations made on members of the same group. Marginal models are often preferred for analyses of population-level effects because the intervention effect coefficient is interpreted as a population-averaged effect. In practice, marginal models are less frequently used than conditional models.⁶

Marginal and conditional intervention effects are equal for identity and log links²² and the distinction between them is only important for link functions such as the logit for binary outcomes. Although some authors have advocated for the log instead of logit link for binary outcomes,²³ this approach is not widely used, possibly because of model convergence problems for some data.^24,25 Alternatively, a modified Poisson approach with log-link and robust standard errors could be used in the GEE framework,²⁶ since it does not suffer from the same convergence problems as the binomial model with log link,²⁷ but it may be less common because of the familiarity of logistic regression among epidemiologists and biostatisticians.

In practice, the question about which of conditional or marginal effects are desired depends on the research question. It is essential to understand the underlying assumptions of each method: conditional models rely on correct specification of untestable aspects of the data distribution, while marginal models rely on a correct definition of the population of interest, which can make it difficult to generalize results to other populations.²⁸ We address each of the two approaches in more detail below.

Conditional Approaches

If the mixed effects model used to estimate conditional effects is misspecified, the estimates are difficult to interpret and, even if regression diagnostics can help,²⁹ standard errors (SEs) are not robust. Fortunately, Murray et al.³⁰ and Fu³¹ have shown that mixed models are robust to substantial violation of the normality assumptions for member- and group-level errors, so long as balance is maintained at the group level. Parameter estimation by restricted maximum likelihood estimation (REML) is preferred to MLE when few groups are available.^32–34 For binary outcomes, alternative methods for specifying the test degrees of freedom have been examined in small sample GRTs and the between-within method is recommended.^32,35

Multiple Levels of Clustering in Conditional Models

GRTs may involve multiple levels of clustering due to repeated measures on individuals or groups or additional hierarchical levels in the design. Murray¹ distinguished between mixed-effects models based on the number of measurements included in the analysis and recommended mixed-effects analysis of variance (ANOVA) or covariance (ANCOVA), or mixed-effects repeated measures ANOVA/ANCOVA, for analyses involving 1 or 2 measurements per person or per group; those models can account for all sources of random variation in such data if they are properly specified.³⁶ However, that is not the case in analyses involving 3 or more measurements per person or per group, where the sources of random variation may be different; instead, such analyses require a random coefficients model in which random trends and intercepts are calculated for each member (in cohort GRT designs) and group (in cohort and cross-sectional GRT designs), average trends and intercepts are calculated for each study arm, and the intervention effect is the net difference in the average study-arm trends.³⁶ Trends are often estimated as linear slopes, but can take another form.

Variable Group Size in Conditional Models

Johnson et al. focused on the analysis of Gaussian outcomes from GRTs with variable group size.³⁷ They compared ten model-based approaches and found that a one-stage mixed model with Kenward-Roger³² degrees of freedom and unconstrained variance components performed well for GRTs with 14 or more groups per study arm. A two-stage model weighted by the inverse of the estimated theoretical variance of the group means and with unconstrained variance components performed well for GRTs with 6 or more groups per study arm. A number of other models resulted in an inflated type I error rate when there was substantial variability in group size.

Marginal Approaches

When the GEE approach is used to estimate marginal effects, unbiased intervention effects can be estimated even if the working correlation structure is incorrect (e.g. using robust SEs via the sandwich estimator), although precision is increased if the working matrix is correct. Where degrees of freedom are limited for the test of interest, as often happens in GRTs, SE estimation is often biased downward and no method corrects for it in all cases, although several have been proposed.^38–44

Multiple Levels of Clustering in Marginal Models

While multilevel clustering is easy to account for in mixed-effects regression, there is less literature for the GEE approach. The alternating logistic regression approach⁴⁵ for binary and ordinal outcomes can be used to account for correlation due to repeated measures on individuals within groups and can be implemented within a GEE framework in both R (the alr package) and SAS (PROC GEE).⁴⁶ The second-order GEE approach which, in contrast to regular GEE, models the working correlation structure as a function of covariates, can be implemented in R ( geepack in R⁴⁷).⁴⁸ For more general working correlation matrices, the user typically needs to perform additional programming in order to provide the appropriate covariance matrix and convergence may not be achieved. In addition, although the intervention effect is unbiased when the marginal model is not correctly specified, the SEs estimated using GEE may be too small. To correct this, a robust sandwich estimator of the variance can be used but such an approach leads to loss of power.⁴⁹ Because of this accuracy-power trade-off, mixed-effects models may be a better option to deal with GRTs involving more than two levels, although the effects estimated in such models are conditional rather than marginal effects.

Variable Group Size in Marginal Models

Although GEE analysis can accommodate variable group size, informative group size can negatively impact efficiency. In this case, Williamson et al.⁵⁰ showed that GEE weighted by group size can correct bias in the estimated intervention effect. This approach is equivalent and less computationally demanding than within-cluster resampling.⁵¹

Advanced GEE Approaches to Improve Efficiency

For binary outcomes, GEE is more conservative (i.e. the intervention effect will be estimated closer to the null) than mixed-effects models.^28,52 Moreover, the SE of the estimated intervention effect is also typically larger when using GEE so that much recent effort has focused on efficient estimation. GEE is most efficient when the true correlation structure of the data is chosen as the working correlation structure. Hin et al. compared multiple selection criteria for the working correlation matrix.⁵³ An alternative approach is augmented GEE (AU-GEE), a method developed for independent data using a causal inference framework,⁵⁴ which has been extended to clustered data.⁵⁵ AU-GEE uses covariate information to improve efficiency in a two-stage approach that specifies a model for the potential outcomes under the treatment not received. AU-GEE is unbiased and robust to misspecification of the potential outcome model, though correct specification improves efficiency. As for the analysis of all trials, only baseline covariates should be included in AU-GEE for the analysis of GRT data because adjustment for post-baseline covariates may lead to bias.⁵⁶ Alternative methods are available to account for post-baseline, time-varying confounding.^57–59

Alternatives to GEE

The quadratic inference function (QIF) method is an alternative to GEE for the estimation of marginal effects. Song et al.⁶⁰ demonstrate that QIF has advantages over GEE: it is more efficient and more robust to outliers; it has a goodness-of-fit test of the marginal mean model and permits straightforward extensions to model selection. In large samples, QIF is more efficient than GEE when the working correlation structure for the data is misspecified.⁶¹ However, the SEs may be under-estimated for small and medium sample size or for variable group size.⁶² More recent work by Westgate^63,64 provides improvements by using a bias-corrected sandwich covariance estimate and by simultaneously selecting the QIF or GEE while selecting the best working correlation structure.⁶⁵ Despite the many attractive properties of QIF, at this time there are few applications in public health.^66–68

A second alternative estimation method is targeted maximum likelihood estimation (tMLE).⁶⁹ tMLE is a maximum likelihood-based G-computation estimator that targets the fit of the data-generating distribution to reduce bias in the parameter of interest. It is based on a machine learning approach that fluctuates an initial estimate of the conditional mean outcome and minimizes a loss function to provide an estimate of the parameter of interest.⁷⁰ The approach has been used in public health^71,72 and shows much promise for GRTs^73,74 because it can improve efficiency by simultaneously accounting for missing data and chance baseline covariate imbalance without committing to a specific functional form.⁷⁵

Permutation Methods

Permutation analysis was introduced for GRTs by Gail et al. for the COMMIT trial.⁷⁶ They found that the permutation test had nominal type I and II error rates across a variety of settings common to GRTs, when the member-level errors were Gaussian or binomial, even when very few heterogeneous groups were randomized to each study arm, and even when the ICC was large, so long as there was balance at the level of the group. Murray et al.³⁰ extended this work, showing that unadjusted permutation tests offer no more protection against confounding than unadjusted model-based tests, while the adjusted versions of both tests perform similarly. The permutation test was more powerful than the model-based test when the data were binomial and the ICC≥0.01. Fu³¹ extended the work to heavy tailed and very skewed distributions and reported similar results.

Li et al. compared model-based and permutation methods in the context of constrained randomization adjusting for group-level covariates. They found that both the adjusted F-test and permutation test maintained the nominal size and had similar power, but cautioned that the randomization distribution must be calculated within the constrained randomization space to prevent inflating the type I error rate.¹⁹

DEVELOPMENTS IN THE ANALYSIS OF ALTERNATIVES TO THE PARALLEL GRT

Stepped Wedge GRT

Both between- and within-group information is available to estimate the intervention effect from a stepped wedge group randomized trial (SW-GRT).^77,78 However, because the control condition is typically observed earlier than the intervention condition, time is a potential confounder and should be accommodated in the analysis of SW-GRTs, typically by accounting for time as a predictor.⁷⁹ As for parallel GRTs, clustering by group must be accounted for, and longitudinal measures on individuals can be accommodated within either the mixed-effects or GEE framework, though more easily using mixed-effects models (see both Multiple Levels of Clustering sections). Conditional approaches are more commonly used in practice and reported on in the methods literature.^79,80 Several authors have highlighted other characteristics specific to SW-GRT including lagged intervention effects⁸¹ and fidelity loss over time.⁷⁹

Network-Randomized GRT

Because the network properties of a network-randomized GRT are primarily used at the design stage,⁸² and because they differ from regular GRTs only in the novel way in which groups are defined, the theory on the analysis of parallel-arm GRTs can be applied to parallel-arm network-randomized GRTs.⁸³ For example, in a ring trial of an Ebola vaccine,⁸³ in which a network was defined as all individuals who had regular physical contact with the incident (index) case of Ebola and in which all contacts received the vaccine (placebo or active), standard GRT methods were used. For network-randomized GRTs in which the intervention is not directly administered to all individuals and in which it is expected that the intervention spreads over the network (e.g. the snowball trials of a HIV prevention intervention for drug users⁸⁴ or a microfinance intervention⁸⁵), methods^86,87 are available to estimate both the direct and indirect effects of the intervention. When network information is available and the outcome of interest is known to be a disseminated process, adjusting for network features such as information on the location of each individual within the network (i.e. group) can improve both the efficiency and power of the analysis.⁸⁸

Pseudo-Cluster Randomized Trial

Teerenstra et al.⁸⁹ compared analytic methods for continuous outcomes in pseudo-cluster randomized trials (PCRT) and Campbell and Walters discussed principles in their recent textbook.⁵ Clustering by the unit of randomization at the first stage (e.g. provider) must be accounted for in both the design and analysis of PCRT. No explicit sample size or analytic methods are known to be available for non-continuous outcomes.

Individually Randomized Group Treatment Trial

Baldwin et al. compared four analytic models for IRGTs and three methods for calculating degrees of freedom.⁹⁰ A multilevel model adapted to reflect clustering in only one study arm, combined with either Satterthwaite⁹¹ or Kenward-Roger³² degrees of freedom, provided better type I error control, better efficiency, and less bias, even with heteroscedasticity at the member level. This finding is consistent with earlier reports by Pals et al.⁹² and Roberts et al.⁹³ More recently, Roberts & Walwyn⁹⁴ and Andridge et al.⁹⁵ considered the circumstance in which members are associated with more than one small group or change agent. Both found that ignoring membership in multiple groups further inflates the type I error rate. Roberts & Walwyn reported that multiple member multilevel models maintained the nominal type I error rate; they also provide sample size and power formulae.⁹⁴

DEVELOPMENTS TO ADDRESS DATA CHALLENGES

Missing Outcome Data

Two recent reviews^6,96 indicate that missing outcome data is common in GRTs, though investigators frequently analyze only available data without accounting for the missing data pattern. When the covariate-dependent missingness (CDM) assumption is plausible, both mixed effects and GEE models provide unbiased estimates of the intervention effect when the CDM covariates are included in an analysis of all available data.^97,98 AU-GEE also can provide unbiased effects by including all CDM covariates in the augmentation component⁵⁵ and has the advantage that all estimates can still be interpreted as marginal effects. Other two-stage approaches such as multiple imputation (MI) or inverse probability weighting (IPW) can provide unbiased intervention effects under certain conditions for more general missing at random (MAR) patterns and may provide increased precision compared to covariate-adjusted conditional or marginal models for CDM.^97,99 Although there is less literature on how to deal with missing not-at-random (MNAR) data,¹⁰⁰ sensitivity analyses are recommended.¹⁰¹ A recent review showed that very few GRTs performed any sensitivity analyses for their missing data assumptions.⁶

To avoid possible type I error, MI should account for the clustered data structure.^102,103 Fixed group effects should not be used due to reduced power.¹⁰⁴ For binary outcomes, Ma et al.¹⁰⁵ and Caille et al.¹⁰⁶ show that the preferred MI method depends on the number of groups and the design effect, and note that bias may arise for some approaches even for CDM missingness. Using group-specific mean imputation may be adequate for continuous outcomes.^98,102 Hossain et al.⁹⁸ show that if the missing data mechanism has an interaction between a covariate predictive of the outcome and study arm, the imputation strategy must account for this interaction to be unbiased.

Whereas MI requires specifying the distribution of the missing data conditional on covariates, IPW requires specifying the probability of being missing depending on covariates. Theoretically, both approaches can be used for any type of outcome and for both CDM and more general forms of MAR mechanisms.⁹⁹ While IPW requires an additional assumption of positivity (all participants have a non-zero probability of being observed), it may be viewed as easier to define, particularly in the presence of non-intermittent missingness.¹⁰⁷ Importantly, and as for MI, if the missing data mechanism has an interaction between a covariate predictive of the outcome and study arm, the weights must be generated by accounting for this interaction in order to be unbiased.¹⁰⁸ Prague et al.^109,110 developed a doubly robust estimator in the context of IPW, which provides an unbiased estimate if either the marginal mean model or the missing data model is correctly specified. They demonstrated that a doubly-robust augmented GEE approach can simultaneously account for both CDM and baseline covariate imbalance in GRTs when the parameter of interest is a marginal effect. Combining MI and IPW is a promising new approach which may have superior performance to IPW or MI alone when there are missing covariates in addition to missing outcomes.¹¹¹

Baseline Imbalance of Covariates

While design strategies such as restricted randomization⁸ can help to achieve baseline covariate balance, they may not be easy to implement (e.g. if group characteristics are unknown in advance) and chance imbalance may arise regardless. In this case, some form of model-based covariate adjustment could be used such as standard multivariate regression for conditional models or AU-GEE for marginal models.⁵⁵ The advantage of AU-GEE in this case is that it is doubly robust in that the consistency of intervention effect estimate requires correct specification of either the marginal mean structure or the treatment model, and it separates covariate adjustment from intervention effect estimation thereby reducing the risk of choosing the adjustment models to obtain the most significant results. The standard multivariate regression adjustment approach does not enjoy either of these benefits.

Alternatively, Hansen and Bowers¹¹² proposed a balancing criterion and studied its randomization distribution in order to simultaneously test for balance of multiple covariates in both RCTs and GRTs. Leyrat et al.¹¹³ suggested to use the c-statistic of the propensity score model to measure covariate balance at the individual level. Leon et al.¹¹⁴ recommended propensity score matching to correct for baseline imbalance; in a simulation study, they report a median 90% reduction in bias. Nevertheless, the Consolidated Standards for Reporting of Trials (CONSORT)¹¹⁵ recommends that the adjustment covariates be specified a priori for primary analyses so that secondary analyses could test sensitivity of the primary findings to adjustment for covariates identified post hoc.

Software

Table 1 identifies three software programs that can be used to analyze data from GRTs. The table is organized around topics considered in the current paper. While none of the three software programs can readily implement both QIF and tMLE for GRTs, the R program offers the most ready-to-use functionality given its broad applicability to the methods cited in the current paper.

Table 1.

Summary of known functions and procedures to analyze GRTs using methods described in the current review.

	Software
Method	SAS	Stata	R
Outcomes analysis of all available data
Mixed-effects models	PROC MIXED	mixed	lme4
	PROC NLMIXED	melogit	nlme
	PROC GLIMMIX	mepoisson
Generalized estimating equations (GEE)	PROC GENMOD1	xtgee	geeglm/geeM
Targeted maximum likelihood (tMLE)	N/A	N/A	N/A2
Quadratic inference function (QIF)	%qif	N/A	qif3
Permutation tests	%ptest	N/A	N/A
Accounting for missing outcomes
Multiple imputation for clustered data	%mmi_impute4	REALCOM Impute	pan
	%mmi_analyze	mi impute4	jomo5
Inverse probability weighting (IPW)	PROC GENMOD6	N/A7	CRTgeeDR
Causal-inference based methods8
Augmented GEE (AU-GEE)	N/A	N/A	CRTgeeDR
Doubly robust AU-GEE	N/A	N/A	CRTgeeDR

Footnotes:

¹. PROC GEE is another option, but is in experimental phase and has limited usefulness for GRTs over and above PROC GENMOD.

². In R, tmle is available for tMLE, but at the time of writing, does not allow for clustering.

³. As of the writing, the authors have been unable to load the package and it only allows equal cluster size, but Westgate has modified the code for GRTs with variable cluster size in the appendix of his paper⁶³

⁴. Only useful for continuous outcomes.

⁵. In R, mice is available for multiple imputation but at the time of writing, does not account for clustering.

⁶. Cannot account for imprecision in the weights.

⁷. xtgee cannot accommodate individual-level weights but only group-specific weights.

⁸. Both of the listed methods are related: AU-GEE accounts for baseline covariate imbalance and doubly robust AU-GEE, an extension of AU-GEE, accounts for both baseline covariate imbalance and missing data.

N/A: not available at the time of writing.

REPORTING OF RESULTS

The CONSORT guidelines for individually randomized trials were extended to GRTs in 2004¹¹⁵ and most journals now require authors to conform to these guidelines. Based on a review of 300 GRTs published between 2000–2008, Ivers et al. reported that 60% and 70% accounted for clustering in the sample size calculation and in the analysis, respectively, 56% used restricted randomization, and most (86%) allocated more than 4 groups per arm.¹⁴ A more recent review of 86 trials published in 2013–2014 showed that 77% and 78% accounted for clustering in the sample size calculation and in the analysis, respectively, and that 51% used some form of restricted randomization.¹⁵

Given concerns about the ethical conduct of GRTs,^116,117 recent reports on conduct and reporting have focused on the ethics of GRTs. For example, Sim and Dawson discuss the challenges associated with obtaining informed consent in GRTs.¹¹⁸ The Ottawa Statement on the ethical design and conduct of GRTs was published in 2012¹¹⁹ with a reevaluation in 2015.¹²⁰

DISCUSSION

In this review, we have summarized many of the most important advances in the analysis of GRTs during the 13 years since the publication of the earlier review by Murray et al.⁷ Many of these developments have focused on developments in marginal model parameter estimation (e.g. augmented GEE, QIF and tMLE) and missing data methods. Some topics that space limitations have prevented include review of recent developments in survival outcomes,^2,121–125 measurement bias,^126,127 validity,^128,129 Bayesian methods,^4,130–132 cost-effectiveness analyses^4,133–136 and mediation analyses to uncover mechanisms of action.^137–140

Through this review, we have sought to ensure that the reader is reminded of the value of well-thought out analysis of GRTs and of keeping up to date with the many recent developments in this area. Pairing this knowledge with our companion review of developments in the design of GRTs,⁸ we hope that our review leads to continued improvements in the design and analysis of GRTs.

APPENDIX: GLOSSARY

Augmented GEE

“Augmenting the standard GEE with a function of baseline covariates.”⁵⁵ These methods adapt semiparametric theory developed by Robins¹⁴¹ and Robins, Rotnitzky, and Zhao¹⁴² for observational studies with time-varying exposures and missing data problems, respectively. They consist of leveraging the estimating equation by a predictor function for counterfactual outcomes under the intervention not received by the group/cluster considered missing.⁵⁵

Baseline covariate balance

The group-level and individual-level covariate distributions are similar in all study arms.¹¹

Choice of balancing criterion

Li et al. describe several balancing criteria to assess how well a GRT is balanced across covariates. These include the “best balance” (BB) metric of de Hoop et al.¹⁴³ the balance criterion (B) of Raab and Butcher,¹¹ and the total balance score introduced by Li et al.¹⁹

Coefficient of variation

A measure of between-group variation, defined in Table 1 of our companion paper.⁸

Cohort GRT design

A cohort of individuals is enrolled at baseline and those same individuals are followed up over time.

Constrained randomization

Refers “to those designs that go beyond the basic design constraints to specify classes of randomization outcomes that satisfy certain balancing criteria, while retaining validity of the design.”¹⁴⁴

Cross-sectional GRT design

A different set of individuals is obtained at each time point.

Designed balance at the group level

When there are equal numbers of groups randomized to each study arm.

Intraclass correlation

A measure of between-group variation, defined in Table 1 of our companion paper.⁸

Covariate-dependent missingness (CDM) assumption

The assumption that “missingness in outcomes depends on covariates measured at baseline, but not on the outcome itself.”⁹⁸

Doubly-robust augmented GEE approach

Combining augmented GEE and IPW, a doubly-robust estimator is obtained, which provides an unbiased estimate if either the marginal mean model or the missing data model is correctly specified.^109,110

Equivalence

Assessing whether the new intervention is equivalent to the comparison intervention.

G-computation estimator

A computational method to estimate causal effect in structural nested models. These models are designed to deal with confounding by variables affected by intervention.¹⁴⁵

Individually Randomized Group Treatment Trials

“Studies that randomize individuals to study arms but deliver treatments in small groups or through a common change agent.”^8,92

Informative cluster size

When the outcome measured is related to the size of the cluster.⁵⁰

Missing at Random (MAR) assumption

Rubin’s (1976) definition is that “data are missing at random if for each possible value of the parameter φ [the parameter of the conditional distribution of the missing data indicator given the data], the conditional probability of the observed pattern of missing data, given the missing data and the value of the observed data, is the same for all possible values of the missing data.”¹⁴⁶

Network-Randomized GRT

“The network-randomized GRT is a novel design that uses network information to address the challenge of potential contamination in GRTs of infectious diseases.”^8,82,84,147

Non-inferiority

When a trial is designed to show that the new intervention is not worse than the comparison intervention.

On treatment analyses

When groups are analyzed “according to the intervention they actually received.”²

Per protocol analyses

When groups “not receiving the correct intervention are excluded.”²

Pseudo-cluster randomized trial

Intervention is allocated to individuals in a two-stage process. “In the first stage, providers are randomized to a patient allocation-mix…. In the second stage, patients recruited to the PCRT are individually randomized to intervention or control according to the allocation probability of their provider.”⁸

Stepped Wedge GRT

“A one-directional crossover GRT in which time is divided into intervals and in which all groups eventually receive the intervention.”^8,78

Superiority

When a trial is designed to establish whether a new intervention is superior to the comparison intervention (e.g., another drug, a placebo, enhanced usual care). However, the statistical test is still two-sided, allowing for the possibility that the new intervention is actually worse than the comparison.

Within-cluster resampling

Randomly sample one observation from each cluster, with replacement. Then analyze this resampled dataset. Repeat this process a large number of times. “The within-cluster resampling estimator is constructed as the average” of all of the resample-based estimates (see Hoffman et al.⁵¹ pp. 1122-3).