Review of Recent Methodological Developments in Group-Randomized Trials: Part 1—Design

Abstract

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have highlighted the developments of the past 13 years in design with a companion article to focus on developments in analysis. As a pair, these articles update the 2004 review.

We have discussed developments in the topics of the earlier review (e.g., clustering, matching, and individually randomized group-treatment trials) and in new topics, including constrained randomization and a range of randomized designs that are alternatives to the standard parallel-arm GRT.

These include the stepped-wedge GRT, the pseudocluster randomized trial, and the network-randomized GRT, which, like the parallel-arm GRT, require clustering to be accounted for in both their design and analysis.

A group-randomized trial (GRT) is a randomized controlled trial in which the unit of randomization is a group, and outcome measurements are obtained for members of the group.¹ Also called a cluster-randomized trial or community trial,^2–5 a GRT is the best comparative design available if the intervention operates at a group level, manipulates the physical or social environment, cannot be delivered to individual members of the group without substantial risk of contamination across study arms, or if there are other circumstances that warrant the design, such as a desire for herd immunity or a need to estimate both the direct and indirect intervention effects in studies of infectious diseases.^1–5

In GRTs, outcomes on members of the same group are likely to be more similar to each other than to outcomes on members from other groups.¹ Such clustering must be accounted for in the design of GRTs to avoid underpowering the study, and it must be accounted for in the analysis to avoid underestimated SEs and inflated type I error for the intervention effect.^1–5

In 2004, Murray et al.⁶ published a review of methodological developments in the design and analysis of GRTs. In the 13 years since, there have been many developments in both areas. We highlight developments in both areas in a 2-part series of articles. In this article (part 1), we focus on developments in design. In the second article (part 2), we focus on developments in analysis.⁷ (The glossary of terms is available as a supplement to the online version of this article at http://www.ajph.org.) As a pair, these articles update the 2004 review. With both articles, we provide a broad and comprehensive review to guide readers to seek out appropriate materials for their own circumstances.

DEVELOPMENTS IN FUNDAMENTALS OF DESIGN

Clustering and the choice between a cohort and a cross-sectional GRT design are fundamental to both the design and analysis of GRTs.

Clustering

In its most basic form, a GRT has a hierarchical structure with groups nested within study arms and members nested within groups. Additional levels of nesting may arise through repeated measures over time or from more complex group structures (e.g., children nested in classrooms nested in schools). When designing and analyzing a GRT, it is necessary to account for the clustering associated with the nested design.^1–5

The intraclass correlation coefficient (ICC), or intracluster correlation coefficient, is the clustering measure most commonly used in power calculations and most commonly reported in published studies.⁸ Eldridge et al.⁹ provide a comprehensive review of ICC definitions and measures in general clustered data for both continuous and binary outcomes, the most commonly reported outcomes in GRTs.^10,11 Although the ICC for continuous outcome measures is well defined and generally well understood,^1–4 Eldridge et al.⁹ highlight some of the challenges for binary outcomes and provide several definitions (Table 1 displays the form most commonly presented in GRT texts).^2,4,5,9 Others compare methods to estimate the ICC of a binary outcome.^12–17 The ICC is not easily defined for rates on the basis of person–time data.^2,4 Recent publications have defined ICC for time-to-event data.^18,19

TABLE 1—

Two Common Measures of Clustering for General Clustered Data for Two Common Types of Outcome

Outcome Measure	ICC, ρa	CV, k	Relationship of ICC to CVb
Continuous
Binary

Note. CV = coefficient of variation; GRT = group-randomized trial; ICC = intraclass correlation coefficient. μ is the overall mean for continuous outcome data; π is the overall proportion for binary outcome data; is the between-group variance; is the within-group variance (i.e., residual error variance). As is common practice, the 2 clustering measures are for general clustered data and do not focus on the GRT design in which the intervention effect is of primary interest (chapter 2 of Hayes and Moulton,² e.g., provides more detail). The intervention parameter of interest in GRTs is typically the following: difference of means for continuous outcomes; difference of proportions; ratio of proportions or odds ratio for binary outcomes; or rate difference or rate ratio for event outcomes.

^aThere are multiple definitions of the ICC for binary outcomes.^12–17 The specific formulation we have provided is 1 of the simplest and most commonly used (e.g., Equation 2.4 of Hayes and Moulton² and Equation 8 of Eldridge et al.⁹).

^bNote that whereas the relationship for binary outcomes is only a function of k and the distributional parameter of interest, π, the relationship for continuous outcomes is a function of both the distributional parameter of interest, μ, and .

The coefficient of variation (CV) is a measure of clustering that is defined for general clustered data when the distributional parameter of interest is a mean, proportion, or rate.^3,17 The CV and ICC for continuous and binary outcomes are associated by a mathematical relationship as a function of the distributional parameter of interest (i.e., mean or proportion) and, for continuous outcomes, of the within-group variance, (Table 1).^2,4 Hayes and Moulton² advocate the CV generally in power calculations; Donner and Klar agree for event data analyzed as rates.³

Because of the central role of clustering in planning GRTs, imprecision in the estimated level of clustering can lead to an underpowered trial. Multiple authors address imprecision, and all focus on the ICC.^20–26 Simultaneously, increasingly more publications are reporting ICCs (e.g., Moerbeek and Teerenstra²⁷ provide a comprehensive list of such articles) to aid the planning of future studies, consistent with the CONSORT (Consolidated Standards of Reporting Trials) statement on GRTs.²⁸

Cohort vs Cross-Sectional Designs

The choice between a cohort and a cross-sectional GRT design (or their combination) is driven by the nature of the research question.¹ The cross-sectional design is preferred when the question is about change in a population¹ or when the time to the outcome is so short as to make a cohort study impractical (e.g., studies involving acute conditions).² For example, to observe enough participants with malaria at 6-month follow-up time points and to be able to draw conclusions about population-level behavior related to malaria treatment choices, Laktabai et al.²⁹ chose a cross-sectional design in which they obtained different population samples at each follow-up time point.

By contrast, when interested in change in specific individuals, or in mediation, the most natural choice is the cohort design, in which a cohort of individuals is enrolled and followed over time.¹ For example, Turner et al.³⁰ chose such a design to study child outcomes in mothers with prenatal depression.

Similarly, the cohort design is usually required to generate event data in individuals.² A combination design could be used whereby the cross-sectional design is augmented by subsampling a cohort of individuals who are followed over time, such as in the COMMIT (Clopidogrel and Metoprolol in Myocardial Infarction Trial) study.³¹ A recent review indicated that the cohort design is the most common GRT design (67% of 75 GRTs).³²

DESIGN OF PARALLEL-ARM GRTS

As for individually randomized controlled trials, the goal of randomization in GRTs is to achieve balance of baseline covariates. In contrast to individually randomized controlled trials, another form of baseline balance applies to GRTs, namely, baseline balance of group sample size. Both forms of balance play a role in the sample size and power calculations that are required to design GRTs.

Baseline Imbalance of Group Sample Size

An imbalance of group sample size means that group sizes are different across the groups randomized in the study, which has implications for statistical efficiency. Donner discussed variation in group size for GRTs for a design stratified by group size.³³ Guittet et al.³⁴ and Carter³⁵ studied the impact on power using simulations, which showed the greatest reduction in power with few groups, high ICC, or both.

Several authors have offered adjustments to the standard sample size formula for a GRT to correct for variability in group size on the basis of the mean and variance of the group size or the actual size of each group.^36–39 Others have offered adjustments on the basis of relative efficiency.^40–43

Candel et al.^40,41 reported that relative efficiency ranged from 1.0 to 0.8 across a variety of distributions for group size, with lower values for higher ICCs and greater variability in group size; the minimum relative efficiency was usually no worse than 0.9 for continuous outcomes. They recommended dividing the result from standard formulas for balanced designs by the relative efficiency for the expected group size distribution, which is a function of the ICC and the mean and variance of the group size.⁴⁰ For binary outcomes, they suggested an additional correction factor on the basis of the estimation method planned for the analysis.⁴¹

You et al.⁴² defined relative efficiency in terms of noncentrality parameters; their measure of relative efficiency was a function of the ICC, the mean and variance of the group size, and the number of groups per study arm. Candel and Van Breukelen⁴³ considered variability not only in group size but also between arms in error variance and the number of groups per arm. They recommended increasing the number of groups in each arm by the inverse of the relative efficiency minus 1. Their estimate of the relative efficiency was a function of the number of groups per study arm, the ICC in each study arm, the ratio of the variances in the 2 study arms, and the mean and variance of the group size.

Consistent across these studies was the recommendation that expectations for variation in group sample size be considered during both the planning stages and the analysis stage. Failure in planning can result in an underpowered study,^40–43 and failure in analysis can result in type I error rate inflation.⁴⁴

Baseline Imbalance of Covariates

Imbalance of covariates at baseline threatens the internal validity of the trial. Yet GRTs often randomize a limited number of groups that are heterogeneous in baseline covariates and in baseline outcome measurements. As a result, there is a good chance of baseline covariate imbalance.^6,45 Restricted randomization strategies such as stratification, matching or constrained randomization can be implemented in the design phase to address this issue.

However, stratification may have limited use in GRTs if there are more than a handful of covariates to balance, because of the small number of groups in most trials.⁴⁶ Pair matching also comes with several disadvantages,⁴⁶ because it affects the proper calculation of ICC⁴⁷ and complicates the significance testing of individual-level risk factors.⁴⁸ More recently, Imai et al. presented a design-based estimator,⁴⁹ which led them to advocate the use of pair matching on the basis of the unbiasedness and efficiency of their estimator. Several others highlighted features of this work,^50–52 including the authors’ power calculation that does not depend on the ICC, thus avoiding the known ICC problem.⁵³

Despite the efficiency gains of pair matching over stratification, a simulation study conducted by Imbens led him to conclude that stratified randomization would generally be preferred to pair matching.⁵⁴ We note that strata of size 4 provide virtually all the advantages of pair matching while avoiding the disadvantages, and may be preferred over pair matching for that reason.

To overcome challenges when trying to balance on multiple, possibly continuous, covariates, Raab and Butcher⁵⁵ proposed constrained randomization. It is on the basis of a balancing criterion calculated by a weighted sum of squared differences between the study arm means on any group-level or individual-level covariate and seeks to offer better internal validity than both pair matching and stratification. The approach randomly selects 1 allocation scheme from a subset of schemes that achieve acceptable balance, identified on the basis of having the smallest values of the balancing criterion.

Carter and Hood⁵⁶ extended this work to randomize multiple blocks of groups and provided an efficient computer program for public use. de Hoop et al. proposed the “best balance” score to measure imbalance of group-level factors under constrained randomization.⁵⁷ In simulations with 4 to 20 groups, constrained randomization with the best balance score was shown to optimally reduce quadratic imbalances compared with simple randomization, matching, and minimization.

Li et al.⁵⁸ systematically studied the design parameters of constrained randomization for continuous outcomes, including choice of balancing criterion, candidate set size, and number of covariates to balance. With extensive simulations, they demonstrated that constrained randomization with a balanced candidate subset could improve study power while maintaining the nominal type I error rate, both for a model-based analysis and for a permutation test, as long as the analysis adjusted for potential confounding.

Moulton⁵⁹ proposed to check for overly constrained designs by counting the number of times each pair of groups received the same study arm allocation. He revealed the risk of inflated type I error in overly constrained designs using a simulation example with 10 groups per study arm. Li et al. further noticed the limitation of overly constrained designs in that they may fail to support a permutation test with a fixed size.⁵⁸ In practice, if covariate imbalance is present even after using 1 of the design strategies described, such imbalance can be accounted for by using adjusted analysis that is either preplanned in the protocol or through post hoc sensitivity analysis.⁷ In summary, constrained randomization seeks to provide both internal validity and efficiency.

Methods and Software for Sample Size

If the ICC is positive, not accounting for it in the analysis will inflate the type I error rate, and the power of the trial will be unknown. If the ICC is estimated as negative, as it can be when the true value is close to zero and sampling error leads to a negative estimate or when there is competition within groups,^1–4,9,60 not accounting for it will reduce the type I error rate so that the test is more conservative, and the power of the trial will be lower than planned.⁶¹ Thus, a good estimate of the ICC is essential for sample size calculation for all GRTs.

One of the simplest power analysis methods often offered for a standard parallel-arm GRT with a single follow-up measurement is to compute the power for an individually randomized trial using the standard formula and to then inflate this by the design effect,⁶² given by . In this formula, is the number of subjects per group and ρ is the ICC.

Unfortunately, this approach addresses only the first of the 2 penalties associated with group randomization that were identified by Cornfield almost 40 years ago⁶³: extra variation and limited degrees of freedom for the test of the intervention effect. To accurately estimate sample size and power for a GRT, it is necessary to also account for the limited degrees of freedom that can arise because of having few groups to randomize. This can be achieved by using appropriate methods detailed in one of the GRT texts rather than using the naïve approach of simply inflating the individually randomized trial sample size by the design effect.^1–5,61

In general, appropriate methods calculate sample size using a variance estimate inflated on the basis of the expected ICC and use a t test rather than a z test to reflect the desired power and type I error rate, with degrees of freedom determined on the basis of the number of groups to be randomized.

In practice, both cross-sectional and cohort GRTs are commonly powered on the basis of a comparison between study arms at a single point in time. Then, for GRTs with cohort designs, the analysis section of the study protocol may state that power will be gained by accounting for the repeated measures design in the analysis. However, methods exist for directly computing power in the case of repeated measures in the context of both cross-sectional and cohort designs.^1,27

Authors have noted that regression adjustment for covariates often reduces both the ICC and the residual variance, thereby improving power.^1,64 Heo et al.⁶⁵ and Murray et al.⁶⁶ provide methods that use data from across the entire course of the study, rather than just comparing 2 means at the end of the study. In practice, the user would require estimates of the variance reduction expected from repeated measures or from regression adjustment for covariates, which could be obtained from previous studies or pilot data.

Methods exist to power GRTs with additional layers of clustering, whether from additional structural hierarchies^1,67–69 or from the repeated measures in the cohort design.^{1,27,64,66,70–73} Konstantopoulos describes how to incorporate cost into the power calculation for 3-level GRTs.⁷⁴ Hemming et al. discuss approaches to take when the number of groups is fixed ahead of time.⁷⁵ Two recent articles focus specifically on binary outcome variables.^13,76 Candel and Van Breukelen examine the effects of varying group sizes in the context of a 2-arm GRT.⁷⁷ Durán Pacheco et al. focus on power methods for overdispersed counts.⁷⁸

Rutterford et al. and Gao et al. summarize a wide array of methods for sample size calculations in GRTs,^79,80 including for GRT designs involving 1 to 2 measurements per member or per group and for designs involving 3 or more measurements per member or per group. A new textbook on power analysis for studies with multilevel data also provides a thorough treatment.²⁷ Previous textbooks on the design and analysis of GRTs devoted at least a chapter to methods for power and sample size.^1–5 A range of software and procedures are available to implement power and sample size calculations for GRTs (Table 2).

TABLE 2—

Software for Sample Size Calculations in Parallel-Arm GRTs

Software	Functionality
PASSa	Sample size calculations for GRTs comparing 2 means (noninferiority, equivalence, or superiority), 2 proportions (noninferiority, equivalence, or superiority), 2 Poisson rates, and a log-rank test
nQueryb	Comparison of 2 means, proportions, and rates
Statac	User-provided command clustersampsi; can compute sample size for continuous, binary, and rate outcomes for 2-sided tests in equal-sized arms
Rd	Package CRTSize for comparing 2 means or 2 binary proportions
SASe	No built-in functionality at this time
Calculator	For some simple designs, parameter values can be plugged into formulas provided in textbooks

Note. GRT = group-randomized trial; PASS = Power and Analysis Software.

^aVersion 15 (NCSS Statistical Software, Kaysville, UT).

^bVersion 7 (Statsols, Boston, MA).

^cVersion 14 (StataCorp LP, College Station, TX).

^dVersion 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria).

^eVersion 9.4 (SAS Institute, Cary, NC).

DEVELOPMENTS IN THE DESIGN OF ALTERNATIVES

Many alternative designs can be used in place of a traditional parallel-arm GRT (Figure 1a). We consider four alternative designs, all of which involve randomization and some form of clustering that must be appropriately accounted for in both the design and analysis (Figure 1, Table 3). Thus, they share key features of the standard parallel-arm GRT, yet all have distinct and different features that are important to understand. In practice, some of these designs are still poorly understood.

FIGURE 1—

Pictorial Representation of Designs for (a) Parallel-Arm GRT, (b) Stepped-Wedge GRT, (c) Network-Randomized GRT, (d) Pseudocluster Randomized Trial (PCRT), and (e) Individually Randomized Group-Treatment (IRGT) Trial

Note. GRT = group-randomized trial. Each pictorial representation is an example of the specific design in which baseline measurements are taken. Other versions of each design exist. All examples show 5 individuals per group. The stepped-wedge GRT is a 1-directional crossover GRT in which time is divided into intervals and all groups eventually receive the intervention, indicated by the shading of the boxes. The design is an example of a “complete design,” that is, every group is measured at every time point. Like parallel-arm GRTs, stepped-wedge GRTs can be either cross-sectional or cohort. In the pseudocluster randomized trial, a group randomized to “intervention” is a group that contains a larger proportion of group members receiving the intervention than does a group randomized to “control”.

TABLE 3—

Characteristics of the Parallel-Arm Group Randomized Trial and of Alternative Group Designs

	One-Stage Randomization			Type of Follow-Up Possible
Design (Abbreviation)	By Group	By Individual	Two-Stage Randomization	Cross-Sectional	Cohort
Parallel-arm group-randomized trial (GRT)	✓	. . .	. . .	✓	✓
Stepped-wedge group-randomized trial (SW-GRT)	✓	. . .	. . .	✓	✓
Network-randomized group-randomized trial (NR-GRT)	✓	. . .	. . .	. . .	✓a
Pseudocluster randomized trial (PCRT)	. . .	. . .	✓	. . .	✓b
Individually randomized group-treatment trial (IRGT trial)	. . .	✓	. . .	. . .	✓c

^aIn the NR-GRT, the index case and its network are usually defined at baseline, and therefore the design is expected to use a cohort design and not allow a cross-sectional design.

^bIn the PCRT, because randomization is undertaken in 2 stages with individuals randomized to intervention or control in the second stage, the design requires that a cohort of individuals be enrolled at study baseline to be followed over time.

^cIn the IRGT trial, individual randomization is performed, and therefore, like the pseudocluster randomized trial, a cohort of individuals is enrolled and followed over time.

Stepped-Wedge GRTs

The stepped-wedge GRT (SW-GRT) is a 1-directional crossover GRT in which time is divided into intervals and all groups eventually receive the intervention (Figure 1b).⁸¹ Systematic reviews indicate increasing popularity.^82–84 Trials recently published a special issue (2015, issue 16) on the design and analysis of SW-GRTs, and many issues of the Journal of Clinical Epidemiology have featured multiple SW-GRT articles (e.g., 2012, 65[12] and 2013, 66[9]).

The rationale for this alternative is primarily logistical: it may not be possible to roll out the intervention in all groups simultaneously,^85–88 although a staggered parallel-arm GRT design could alternatively be used in which blocks of groups are randomized to intervention or control instead of all groups eventually receiving the intervention as in the SW-GRT.^89–91 Others propose a SW-GRT for ethical and acceptability reasons because all groups eventually receive the intervention.⁸² This second argument has been discounted because the intervention could be delivered to all control groups at the end of a parallel-arm GRT design,^88,92 often earlier than would be the case in a SW-GRT.⁹³ When SW-GRTs are conducted in low-incidence settings, Hayes et al. emphasized that the order and period of intervention allocation is crucial.⁹⁴

For the parallel-arm GRT, design choices include cross-sectional⁸² versus cohort,⁹⁵ with most SW-GRT methodological literature focused on cross-sectional designs, although most published SW-GRTs are cohort designs.⁹⁶ An additional variation is that of complete versus incomplete SW-GRTs defined according to whether each group is measured at every time point.⁹⁰ Regardless of the specifics of the SW-GRT design, it is important to consider the possible confounding and moderating effects of time in the analysis.^{85,90,97–99} Failure to account for both, if they exist, will threaten the internal validity of the study.

Cross-sectional SW-GRT sample size formulas are available for complete and incomplete designs.^90,100–103 Hemming et al. provide a unified approach for the design of both parallel-arm and SW-GRTs and allow multiple layers of clustering.⁹⁰ Cohort SW-GRT sample size calculation relies on simulation.^97,104 Recent work on optimal designs shows that, for large studies, the optimal design is a mixture of a stepped-wedge trial embedded in a parallel-arm trial.^105,106 Moerbeek and Teerenstra devote a chapter to sample size methods for SW-GRTs.²⁷

Network-Randomized GRTs

GRTs have historically been used to minimize the contamination between study arms; such contamination is also called “interference.”¹⁰⁷ This contamination may give rise to a network of connections between individuals both within and between study arms. The latter is of particular relevance to GRT design because it leads to reduced power, although sample size methods exist to preserve power and efficiency.¹⁰⁸

The network-randomized GRT is a novel design that uses network information to address the challenge of potential contamination in GRTs of infectious diseases.^109–111 In such a design, groups are defined as the network contacts of a disease (index) case, and those groups are randomized to study arms. Examples include the snowball trial and the ring trial, each with a distinct way to deliver the intervention. In the snowball trial, only the index case directly receives the intervention; the index is then encouraged to share the intervention with her or his contacts (e.g., see Latkin et al.¹⁰⁹ for such a trial of HIV prevention in injection drug users). In the ring trial, “rings” of contacts of the index case are randomized to receive the intervention (Figure 1c). This design has been used to study foot-and-mouth disease,¹¹² smallpox,¹¹³ and Ebola.¹¹⁴ For the same sample size, ring trials are more powerful than are classical GRTs when the incidence of the infection is low.¹¹⁵

Pseudocluster Randomized Trials

In GRTs where all members of the selected groups are recruited to the study, study participants are expected to be representative of the underlying population and, as a result, selection bias is expected to be minimal. By contrast, GRTs with unblinded recruitment after randomization are at risk for selection bias. For example, consider a GRT used to evaluate the effect of a behavioral intervention delivered by providers in the primary care setting. If a provider is first randomized to a study arm and then prospectively recruits participants, she or he may differentially select participants depending on whether she or he is randomized to the intervention or control arm.¹¹⁶

To reduce the risk of such selection bias, Borm et al. introduced the pseudocluster randomized trial (PCRT) to allocate intervention to participants in a 2-stage process.¹¹⁷ In the first stage, providers are randomized to a patient allocation mix (e.g., patients predominantly randomized to intervention vs patients predominantly randomized to control). In the second stage, patients recruited to the PCRT are individually randomized to intervention or control according to the allocation probability of their provider (e.g., 80% to intervention vs 20% to intervention; Figure 1d).

An obvious threat to a PCRT design is that the same providers are asked to implement both the intervention and the control arms, depending on which patient they are seeing. Concerns about contamination are a common reason to randomize providers (i.e., group randomization) so that they deliver either the intervention or the control but not both. The PCRT design would not be appropriate if there are concerns about contamination and if they exceed concerns about selection bias.

In 2 published cases, providers were blinded to the 2-stage form of randomization and instead assumed that patients were individually randomized to the intervention arm with equal probability.^118,119 Later publications indicate that the PCRT design did well at balancing contamination and selection bias in both studies.^120–122

Borm et al. provide sample size calculations for continuous outcomes.¹¹⁷ The clustering by provider (or unit of first-stage randomization) must be accounted for in both the design and analysis. No explicit sample size methods are known to be available for noncontinuous outcomes. Moerbeek and Teerenstra devote a chapter to sample size methods for PCRTs.²⁷

Individually Randomized Group-Treatment Trials

Pals et al.¹²³ identified studies that randomize individuals to study arms but deliver interventions in small groups or through a common change agent as individually randomized group-treatment (IRGT) trials, also called “partially clustered or partially nested designs” (Figure 1e).^72,124 Examples include studies of psychotherapy,¹²⁵ weight loss,¹²⁶ and reduction in sun exposure.¹²⁷ Clustering associated with these small groups or change agents must be accounted for in the analysis to avoid type I error rate inflation.^{72,123,124,128,129} Even so, this accounting appears to be rare in practice.^{123,130–133}

Recent articles have reported sample size formulas for IRGT trials with clustering in only 1 study arm, both for balanced^{72,123,128,134} and unbalanced designs.^77,128 Moerbeek and Teerenstra devote a chapter to sample size methods for IRGT trials focused on methods with clustering in either 1 or both arms.²⁷ Roberts addresses sample size methods for IRGT trials in which members belong to more than 1 small group at the same time or change small groups over the course of the study.¹³⁵ Both features have been shown to increase the type I error rate if ignored in the analysis.^135,136

CONCLUSIONS

We have summarized many of the most important advances in the design of GRTs during the 13 years since the publication of the earlier review by Murray et al.⁶ Many of these developments have focused on alternatives to the standard parallel-arm GRT design as well as those related to the nature of clustering and its features in all the designs presented. Space limitations have prevented us from including recent developments involving pilot and feasibility GRTs; designs to improve efficiency, such as factorial and crossover GRTs; and group designs, such as cutoff designs and regression discontinuity applied to groups. Interested readers are directed to the recently launched peer-reviewed journal Pilot and Feasibility Studies and related references ^4,137; to a recent methodological review of efficiency improvements for GRTs by Crespi,¹³⁸ including factorial and crossover GRTs; to additional developments in crossover GRTs,^139,140 including a recent review by Arnup et al.¹⁴¹; to additional reflections on the factorial GRT by Mdege et al.¹⁴²; and to cutoff design references by Pennell et al.¹⁴³ and by Schochet.¹⁴⁴

With this review, we have sought to ensure that the reader is reminded of the value of good design and gains knowledge in the fundamental principles of a range of recent and potentially beneficial design strategies. Pairing this knowledge with our companion review of developments in the analysis of GRTs,⁷ we hope that our work leads to continued improvements in the design and analysis of GRTs.