Statistical properties of minimal sufficient balance and minimization as methods for controlling baseline covariate imbalance at the design stage of sequential clinical trials

Abstract

When the number of baseline covariates whose imbalance needs to be controlled in a sequential randomized controlled trial is large, minimization is the most commonly used method for randomizing treatment assignments. The lack of allocation randomness associated with the minimization method has been the source of controversy, and the need to reduce even minor imbalances inherent in the minimization method has been challenged. The minimal sufficient balance (MSB) method is an alternative to the minimization method. It prevents serious imbalance from a large number of covariates while maintaining a high level of allocation randomness. In this study, the two treatment allocation methods are compared with regards to the effectiveness of balancing covariates across treatment arms and allocation randomness in equal allocation clinical trials. The MSB method proves to be equal or superior in both respects. In addition, type I error rate is preserved in analyses for both balancing methods, when using a binary endpoint.

1 |. INTRODUCTION

In randomized controlled clinical trials, variables other than the treatment effect, known as covariates, are not randomized and can confound the relationship between the treatment and the outcome. McEntegart¹ argues that controlling imbalance in covariates is important for maintaining the credibility of a trial, making interpretations easier than adjustment alone, and protecting against model misspecification. Important baseline covariates may confound the estimation of treatment effects and, therefore, should be prespecified based on strong clinical rationale and adjusted for in the primary analysis.² In addition, serious baseline covariate imbalances may lead to suspicions of selection bias.³ Therefore, to reduce confounding and isolate treatment effect, controlling imbalance in important baseline covariates is desirable.^4,5

When the number of covariates is small, stratified restrictive randomization methods are commonly used to control covariate imbalances.¹ When the number of baseline covariates is large, stratification can become impractical, and the minimization method, independently proposed by Taves⁶ and Pocock and Simon,⁷ can be used to control imbalance.^8,9 The minimization method effectively controls the marginal imbalances of many categorical baseline covariates. However, use of minimization for subject randomization in clinical trials has been criticized because (for a given subject in a study) it does not preserve complete allocation randomness,^10,11 namely equal or even near-equal probability of allocation to treatment arms. The method has also been criticized due to the potential inflation of type I error rate when applied to trials with unequal allocations.¹² However, Kuznetsova and Tymofyeyev have developed a method for addressing this issue.¹³

The Minimal Sufficient Balance (MSB) method proposed by Zhao et al, as an alternative to minimization, aims to preserve treatment allocation randomness while preventing serious imbalances in a large number of baseline covariates.¹⁴ MSB’s ability to control covariate imbalance while preserving a high level of allocation randomness has been demonstrated in simulation studies and real trial experiences.^14–16 However, a direct, quantitative comparison of statistical properties between MSB and minimization has not been reported in a realistic trial situation where biased coin treatment assignments are made. This study does not aim to establish the MSB method as the ultimate method for controlling covariate imbalances, but to explore the way the MSB method’s properties compare to the most commonly used method of controlling imbalance on a large number of covariates, which is minimization. Other methods exist for controlling covariate imbalances, such as those of Begg and Iglewicz¹⁷ and Atkinson,¹⁸ but none of these methods are commonly used in practice, and thus are not the focus of this study. In Section 2, differences between MSB and minimization are discussed based on their philosophical objectives as well as technical aspects. In Sections 3 and 4, properties of the minimization method and MSB are compared via simulations based on the NINDS rt-PA trial data,¹⁹ followed by a discussion in Section 5.

2 |. BACKGROUND

The fundamental basis for statistical inference from randomized control trials is the true random allocation of subjects. In spite of its vast popularity, minimization has had a mixed response from researchers in the field of statistics.^14,20–23 The frequent adoption of minimization as a method of promoting covariate balance in clinical trials is problematic because minimization does not preserve a strong rate of allocation randomness.^11,24 The three criteria to determine the effectiveness of a method in promoting covariate balance in clinical trials include allocation randomness, effectiveness in controlling imbalance on specified covariates, and ease of implementation. Thus, the background section includes an exploration of the two methods that comprise this study followed by a consideration of these three criteria.

2.1 |. MSB implementation

Implementation of MSB simulations begins with a 20 subject run-in period where the assignments are completely random. This run-in period can be modified based on target sample size. For all subsequent assignments, two treatments are considered, A and B. Assume a subject is prepared to be randomized to a treatment arm. Let n_A and n_B be the number of subjects who have received treatment A or B, respectively. If covariate j is continuous, a t test is used to determine the imbalance. Let x_j be the value of the jth covariate for the incoming subject. Let ${\overline{x}}_{j\text{A}}$ and ${\overline{x}}_{j\text{B}}$ be the means of covariate j among the subjects who had previously received A or B, respectively. Let t_j and t_j* be the test statistic and positive critical values of the independent two-sample t test comparing the values of covariate j within treatments A and B, respectively. Let p_j be the P-value associated with the t test comparing ${\overline{x}}_{j\text{A}}$ and ${\overline{x}}_{j\text{B}}$ and γ_j be the minimum tolerable P-value for covariate j.

As a result of these statistics, no vote is made on covariate j if p_j ≥ γ_j. Otherwise, a vote is made in favor of A if the following criteria are met: (a) t_j < −t_j* and $x_j>{\overline{x}}_{j\text{A}}$ or (b) t_j > t_j* and $x_j>{\overline{x}}_{j\text{A}}$. Otherwise, if: (a) t_j < −t_j* and $x_j>{\overline{x}}_{j\text{A}}$ or (b) t_j > t_j* and $x_j>{\overline{x}}_{j\text{A}}$, a vote is made for treatment B. If none of these conditions is met, no vote is made. Votes are tabulated after all covariates have been observed to determine whether one treatment should be favored in order to reduce covariate imbalance across treatment arms.

If, on the other hand, covariate j is categorical with G levels, a Chi-square test is used to determine the level of imbalance. Let n_jgA and n_jgB be the number of subjects who have previously been polychotomized in category g of covariate j in treatments A and B, respectively. Let E_jgA and E_jgB be the expected number of subjects in category g of covariate j under the null hypothesis for treatments A and B, respectively. Let p_j be the P-value associated with the Chi-square test statistic with (G − 1) degrees of freedom comparing the levels of covariate j and γ_j be the minimum tolerable P-value of covariate j.

Assume that the next subject is in category g of covariate j. As a result of these statistics, no vote is made on covariate j if p_j ≥ γ_j. Otherwise a vote is made for A if the following criteria are met: p_j < γ_j and E_jgA > n_jgA. Otherwise, if p_j < γ_j and E_jgB > n_j,gB, a vote is made against A.

After all of the baseline covariates have been examined, votes are tabulated; and if there are more votes for A than against A, the next subject is assigned to treatment A with a probability of ξ, where ξ is the biased coin probability. If there are more votes against A than for A, the next subject is assigned to treatment B with probability ξ. Otherwise, the allocation probability is 0.5. Although Zhao et al¹⁴ suggest using ξ between 0.65 and 0.7, other values are possible in the range of (0.5, 1), depending on the investigator’s comfort with imbalances in baseline covariates and the desired level of allocation predictability. The values of γ₁, … , γ_G indicate the lowest allowable P-values for each covariate. Smaller P-values allow greater imbalance in the covariates; however, they promote more completely random treatment assignments. Zhao et al¹⁴ suggest 0.2 ≤ γ₁, … , γ_G ≤ 0.3.

2.2 |. Minimization implementation

In order to implement Pocock and Simon’s minimization,⁷ continuous covariates must first be polychotomized into two or more categories. Thus, all imbalances is measured in the context of the new, categorical variables. Consider two treatments, A and B, where controlling imbalance on J categorical variables is desired. The first subject is assigned to either to treatment A or B with probability of 0.5. For all subjects following the first subject, total imbalance under hypothetical assignment to treatment A or B must be calculated before the allocation probabilities are determined. Prior to assignment, let x_ij represent the value of the jth covariate of the ith subject. Let x_Tij represent the value of the jth covariate of the ith subject in group T (where T is A or B). To calculate the absolute imbalance if subject k is assigned to treatment T, the following equation is used, where Im (k,T) is referred to as the total imbalance if subject k is assigned to treatment T:

$$\text{Im}(k,T)=\sum _{j=1}^J\left|\sum _{i=1}^{n_T}{I}_{\left\{x_{Tij}=x_{kj}\right\}}-\sum _{i=1}^{n_{\overline{T}}}{I}_{\left\{x_{\overline{T}ij}=x_{kj}\right\}}\right|,$$

where I_{} is the identity function and $\overline{T}$ is the opposite treatment than T (ie, B or A). Thus, to determine the allocation probability, first it must be determined if the total imbalance can be reduced by assigning subject k to a particular treatment. Thus, for k > 1, if Im(k, A) < Im(k, B), then subject k is assigned to treatment A with probability ξ, where ξ is the biased coin probability. If Im(k, A) > Im(k, B), then subject k is assigned to treatment B with probability ξ. Otherwise if Im(k, A) = Im(k, B), subject k is assigned to treatment A with probability 0.5.

2.3 |. Allocation randomness

Preserving allocation randomness is paramount to any randomization method because the premise of most statistical inference methods assumes data represent random samples where each subject is equally likely to be assigned to each of the treatments in the trial and subjects’ assignments are independent of one another. Thus, all forms of randomization may have implications for analysis. It has been advocated to use permutation tests,^25,26 however, while permutation tests are assumption-free, they are not practical for complex modeling situations.²⁷

Many clinicians conflate concealment of allocation with randomness of allocation.²⁸ These concepts must be distinguished because randomization is the basis for inference. Random allocation guarantees asymptotic balance on all measured and unmeasured covariates; thus, any departure from allocation randomness can bias the treatment effect due to imbalances in measured and unmeasured covariates. The trade-off between allocation randomness and covariate balance exists in all randomization methods; nonetheless, the fact that randomization is necessary for inference must not be dismissed.¹⁴ Zhao et al¹⁴ and Sajobi et al¹⁵ have shown that MSB has a high proportion of completely random treatment assignments (over 50%) due to the method’s ability to correct only for major covariate imbalances. The correct guess probability is also used to measure allocation randomness. The correct guess probability is the probability of successfully guessing the treatment assignment for an unblinded clinician who understands the randomization technique.²⁹ Thus, if α is the proportion of completely random treatment assignments, the correct guess probability is G = (0.5)α + (1 − α)ξ.

Frequent use of constrained randomization, when only minor covariate imbalances are present, is philosophically problematic because the space of all potential randomization schemes has often been markedly reduced, making it sensitive to selection bias. It has been stated and shown that true minimization is not randomization, because the method calls for deterministic assignment for each subject (to the arm that would yield the most balance). As a result, many “randomized” clinical trials are not truly randomized, yet are analyzed as if they were.^10,30 Minimization by design “forfeits the theoretical certainty that is guaranteed by randomization and instead relies on the potentially fallible judgment of statisticians and clinicians”.²⁰ Properties of minimization using a biased coin do not significantly improve allocation randomness, especially when biased coin probabilities as high as 0.85 are often used in practice.²⁶ Under completely random design, the distribution of P-values comparing the distribution of covariates among treatment groups is expected to be marginally uniform (although they behave differently from independent uniforms); however this may not be the case in minimization-based clinical trials due to the lack of randomization inherent in minimization.¹² Although statistical tests comparing covariates across treatment arms are in disfavor, such comparisons are often used in evaluating the validity of a trial.²⁵

Preserving allocation randomness is particularly important in trials where the treatment assignments are not able to be blinded. Due to the lack of completely random treatment assignments, selection biases can conceivably be found in minimization-based trials. One other major role of randomization is to eliminate the ability to predict future assignments,^14,30–32 but it is possible that this feature may be lost almost entirely when minimization is employed. Although this is a potential drawback of minimization, it is difficult to determine whether selection bias actually occurs in minimization-based trials. Nonetheless, the potential for selection bias should be a concern to those who use this randomization method. Even when a biased coin is used for randomization, an unblinded clinician can have a probability of correctly guessing a subject’s treatment assignment which is bounded only by the biased coin probability. It should be noted that some issues related to selection bias may not apply in multicenter trials where randomization is not stratified by center because investigators may only know the sequence of treatment assignments at their center. Nonetheless, decreased randomness can seriously impact the imbalance in measured and unmeasured covariates.

2.4 |. Effectiveness in controlling imbalance on specified covariates

Minimization was designed to control imbalance in categorical covariates and can do so effectively.^6,7 In fact, Burman³³ showed that when minimization is used, marginal covariate imbalance is of the order O_p 1. Nonetheless, when minimization is used with stratified continuous covariates (as is common in clinical trials^34–36), imbalance within strata is of the order ${\text{O}}_{\text{p}}(\sqrt{n})$.²⁷ Both MSB and completely random design result in marginal imbalance that is of the order ${\text{O}}_{\text{p}}(\sqrt{n})$.²⁷ However, covariate-adaptive methods of covariate balance are generally recommended for use in small and medium-sized trials.¹⁴ Thus, asymptotic results should be considered alongside results from simulation studies to fully compare the properties of these methods with respect to covariate balance when covariate-adaptive randomization is used in practice. MSB has been shown in simulation studies to control imbalance in categorical and continuous variables and it has the advantage of controlling imbalance on continuous variables without having to polychotomize them.¹⁴

Although Taves⁶ and Pocock and Simon⁷ claim minimization is effective at limiting the imbalance between covariates across treatment groups, there are several issues that prevent it from working consistently. First, minimization uses a mixed target because it attempts to minimize the total imbalance, rather than the imbalance in each covariate. In addition, due to the need to polychotomize (eg, dichotomize, trichotomize) all covariates, minimization often does not lead to balance for continuous covariates, because the method is sensitive to the threshold(s) used for the continuous variable.^21,37 This issue is a particularly important consideration in clinical trials because many covariates are continuous. In addition, when little information is known about the distribution of the covariates, polychotomization can be done in a way that does not reflect the distribution of the covariate. Even though the randomization methods may balance the polychotomized version of the continuous variables, imbalance is generally assessed on the original scale, which can result in bias. The MSB method, however, controls imbalance in a way that does not depend on the distribution of the variable. As a result, no covariate information is lost in the process of assessing covariate information. No theoretical or simulation-based comparison of the performance of the minimization and MSB methods has been performed to examine the ability of the methods to balance on 10 or more continuous and categorical covariates.

2.5 |. Ease of implementation

One of the challenges with implementing minimization or any other method requiring stratification is that imbalance becomes less efficient when continuous covariates are recoded as categorical covariates causing imbalance reduction in the categorical version of the variable. This process may not lead to desired balance in the continuous variable. The MSB method does not require polychotomization and can use a t test, Wilcoxon Rank Sum test, Chi-square test, or Fisher’s Exact Test to assess the balance between the treatment arms. Some Bayesian adaptive methods require frequent use of a statistician in the allocation process throughout the trial. However, once the thresholds for imbalance (minimization) and the weights of the covariates (MSB and minimization) are specified, the allocations could be and have been successfully automated.^15,16

3 |. METHOD

The MSB method for covariate balance¹⁴ is a strategy in which biased coin treatment assignments are made only in the presence of a serious imbalance that could be reduced via biased coin treatment assignment. Otherwise, completely random assignments are made. The MSB method is flexible to allow for a variety of methods of measuring imbalance. Typically, the P-value is used as a measure of imbalance due to its prevalence in the literature for determining balance.¹⁴ A cautionary note is that P-values of covariate imbalance are intended to be used as a tool for measuring imbalance, but not for inference in a clinical trial. Thus, P-values should not be reported in Table 1 for any given study. Because this article focuses on two-arm, equal allocation trials, it is appropriate to use a t test to determine balance for continuous variables and a Chi-square (or Fisher’s exact) test to determine balance for categorical variables.

TABLE 1.

Results from simulations controlling imbalance in five covariates

Observed P-values in the original study
	Clinical center	NIHSS	Age	OTT	Glucose	Stroke subtype	Sex	Fibrinogen	Weight	Systolic BP	Diastolic BP
	.999	.140	.029	.866	.780	.073	.627	.181	.011	.597	.281
Minimization
P-Value percentiles	Included in randomization method					Not included in randomization method
	Clinical center	NIHSS	Age	OTT	Glucose	Stroke subtype	Sex	Fibrinogen	Weight	Systolic BP	Diastolic BP
2.5%	.469	.223	.225	.266	.115	.047	.026	.023	.035	.026	.026
5%	.601	.300	.303	.336	.178	.082	.052	.051	.065	.053	.050
10%	.733	.397	.386	.422	.261	.143	.105	.104	.125	.100	.098
Median	.962	.737	.721	.760	.653	.542	.500	.507	.526	.499	.504
Minimal sufficient balance
	Included in randomization method					Not included in randomization method
	Clinical center	NIHSS	Age	OTT	Glucose	Stroke subtype	Sex	Fibrinogen	Weight	Systolic BP	Diastolic BP
2.5%	.210	.268	.266	.269	.261	.037	.030	.031	.033	.029	0.26
5%	.269	.297	.295	.293	.291	.071	.057	.055	.063	.056	.052
10%	.314	.335	.330	.329	.320	.131	.103	.108	.114	.106	.099
Median	.666	.629	.610	.623	.623	.533	.500	.504	.522	.494	.501

3.1 |. Simulation study

Simulations for studying the characteristics of the MSB and minimization were modeled after a dataset from the NINDS rt-PA Stroke study obtained from the National Technical Information Service.¹⁹ The cohort of 624 subjects in that study was included in the development of the simulations. Covariates included from this cohort are three categorical variables (clinical center, sex, and stroke subtype) and eight continuous variables (National Institute of Health Stroke Scale [NIHSS], age, onset-to-treatment time, glucose level, fibrinogen level, weight, systolic blood pressure, and diastolic blood pressure). In the original study, only imbalances in onset-to-treatment time (0–90 minutes vs 91–180 minutes) and clinical center were controlled using a stratified permuted block randomization scheme. Although several outcomes were measured in the original dataset, for the purposes of this study, the pattern set out by Johnson³⁸ will be followed by classifying scores of 95 or 100 on the Barthel index, ≤1 on the NIHSS and the modified Rankin scale, and 1 on the Glasgow outcome scale as indicative of a favorable outcome. Data that support the findings of this study can be purchased from the National Technical Information Service (http://www.ntis.gov/).

For this simulation study, the biased coin probability used is 0.65 and the imbalance control limit P-value (or P-value cutoff) is .3, unless otherwise specified. When applying minimization, continuous covariates were polychotomized according to quartiles. This was done to allow minimization to perform as well as possible in the absence of a clinician input.

Because methods used to analyze data assume completely random design, comparisons included permutation tests to examine whether important distributional assumptions were violated by the randomization methods used in the study.³⁹ Thus, permutation tests were performed to examine whether the desired type I error rates of 0.05 for a two-sided hypothesis test were preserved for simulations involving minimization and MSB methods.

In the calculation of rate of false-positive trials, adjusting for covariate effects used to balance with MSB or minimization, the outcome associated with each simulation corresponding to the treatment assignment for each individual was binary. Wald-type 95% confidence intervals were generated for empirical type I error rates in the course of the simulations for both methods.

3.2 |. Distribution of covariates included in the randomization method

Under completely random design, it is expected that the distribution of the P-values for all covariates are marginally uniformly distributed between 0 and 1. Not all subjects have equal probability of being in the treatment or the control groups because both the MSB and minimization method use biased coin assignments, Thus, this study investigates the distribution of the P-values for the covariates included in the randomization methods to better understand the impact that biased coin treatment assignments have on the distribution of specified covariates across treatment groups. The distribution of covariates were evaluated by considering 5000 simulations, where clinical center, NIHSS, age, onset-to-treatment time, and glucose levels were included in the randomization methods and stroke subtype, sex, fibrinogen, weight, systolic blood pressure, and diastolic blood pressure were only observed, without being included in the randomization methods. A P-value under the null hypothesis of balance between treatments on each of these 11 covariates was calculated using t tests or Chi-square tests, as appropriate, for each of the 5000 simulations.

3.3 |. Effect of a large number of covariates on covariate imbalance

Due to philosophical and real data arguments suggesting that minimization does not perform well with a large number of covariates (eg, 10 or more),⁷ this study explores the effect of having a large number of covariates on the ability to control imbalance in covariates during randomization for each of the methods considered in this study. P-Values were used to determine imbalance and performance of both methods in reducing covariate imbalance. As a result, a P-value under the null hypothesis on each of these 11 covariates was calculated using t tests and Chi-squared tests, as appropriate, for each of the 1000 simulations. Fewer simulations were considered with 11 covariates due to the computational challenges posed by the sample size and the 11 covariates.

3.4 |. Impact of biased coin probability on allocation randomness and covariate imbalance

To investigate the potential impact of selection bias, the proportion of pure random assignments under biased coin probabilities of 0.55, 0.6, 0.65, 0.7, 0.8, 0.9, and 1 and the overall covariate balance was assessed using MSB and minimization. For each biased coin probability considered, 1000 simulations were performed using each method. The correct guess probability was calculated to determine the potential for selection bias.²⁹ For each simulation study, the fifth percentile boundary on the balance test (t test, or Chi-square test, for imbalance across treatment arms, as appropriate) for each covariate included in the randomization method was considered to assess the effectiveness of the method at controlling covariate imbalances across treatment arms. The covariates considered for this part of the analysis include: Clinical center, NIHSS, age, onset-to-treatment time, and glucose level.

3.5 |. Impact of P-value on MSB

To investigate the potential impact of the P-value cutoff in the MSB method on allocation randomness and covariate balance, the proportion of pure random assignments under P-value cutoffs of 0.1, 0.2, 0.3, and 0.4 and the overall covariate balance were assessed using MSB. For each biased coin probability considered, 1000 simulation studies were conducted. The correct guess probability was calculated to determine the potential for selection bias.²⁹ For each simulation study, the fifth percentile boundary on the balance test (t test, or Chi-square test, as appropriate) for each covariate included in the randomization method was considered to assess the effectiveness of the method at promoting balance across treatment arms. The covariates that were considered for this part of the analysis include: Clinical center, NIHSS, age, onset-to-treatment time, and glucose level.

4 |. RESULTS

4.1 |. Distribution of covariates included in the randomization method

In general, the P-values to measure covariate balance tend to be larger for MSB for those variables included in the randomization method compared to minimization, especially for continuous variables. The 2.5th, 5th, 10th, and 50th percentiles of P-values comparing covariate distributions across treatment arms from the simulations demonstrate superior balance in MSB for a biased coin probability of .65 (see Table 1). For example, the 2.5th, 5th, and 10th percentiles of P-values for glucose level for minimization were .115, .178, and .261, respectively, and for MSB were .261, .291, and .320, respectively. Values for MSB are consistently close to or greater than the imbalance threshold of .3.

4.2 |. Effect of having a large number of covariates on covariate imbalance

Minimization performs poorly in controlling covariate imbalance, particularly for continuous variables. This can be seen by considering 2.5th, 5th, 10th, and 50th percentiles of P-values from the simulations (see Table 2). The analysis found major differences in the performance of the methods. For example, the 2.5th, 5th, and 10th percentiles of P-values for glucose levels using minimization were .060, .102, and .159, respectively, and for MSB were .256, .283, and .311, respectively. Minimization is inferior at controlling imbalance when the number of covariates becomes large, whereas MSB controls imbalance more consistently regardless of the number of covariates when biased coin probability is fixed. The MSB method had some P-values for imbalance below .2 for categorical variables when the number of categories is large (clinical center and stroke subtype) at the 2.5th percentile; however, at the 5th percentile, all of them were above .2, which is generally acceptable and considered as a minor covariate imbalance. Nonetheless, the performance of the covariates overall are better for MSB than for minimization when biased coin probability is fixed. It should be noted that minimization balanced categorical variables to a much greater degree than continuous variables, resulting in greater P-values than MSB for these variables. However, overall balance is superior using MSB.

TABLE 2.

Results from simulations controlling imbalance in 11 covariates

Observed P-values in the original study
	Clinical center	NIHSS	Age	OTT	Glucose	Stroke subtype	Sex	Fibrinogen	Weight	Systolic BP	Diastolic BP
	.999	.140	.029	.866	.780	.073	.627	.181	.011	.597	.281
Minimization
P-Value boundary	Included in randomization method
	Clinical center	NIHSS	Age	OTT	Glucose	Stroke subtype	Sex	Fibrinogen	Weight	Systolic BP	Diastolic BP
2.5%	.257	.151	.118	.152	.060	.231	.229	.097	.138	.125	.102
5%	.376	.194	.177	.229	.102	.289	.338	.143	.183	.185	.152
10%	.512	.279	.249	.334	.159	.411	.457	.225	.263	.269	.225
Median	.894	.687	.612	.706	.566	.812	.804	.619	.630	.641	.633
Minimal sufficient balance
	Included in randomization method
	Clinical center	NIHSS	Age	OTT	Glucose	Stroke subtype	Sex	Fibrinogen	Weight	Systolic BP	Diastolic BP
2.5%	.158	.263	.261	.260	.256	.156	.256	.254	.244	.241	.250
5%	.249	.288	.286	.300	.283	.226	.288	.278	.288	.267	.283
10%	.318	.327	.328	.336	.311	.293	.330	.325	.326	.313	.324
Median	.666	.618	.615	.604	.617	.587	.651	.624	.632	.632	.634

4.3 |. Permutation tests

Means and Wald-type confidence intervals for the empirical type I error rates of treatment effect from simulations involving five covariates in the randomization method resulted in 95% confidence intervals of .048, (.042, .053) and .052, (.046, .058), for minimization and MSB, respectively. Means and Wald-type confidence intervals for the empirical type I error rates of treatment effect simulations involving 11 covariates in the randomization method resulted in 95% confidence intervals of .046, (.033, 0.059) and .054, (.040, .068), for minimization and MSB, respectively. These results suggest that the nominal type I error rate of 5% was preserved in general for both MSB and minimization.

4.4 |. Impact of biased coin probability on allocation randomness and covariate imbalance

Imbalance and allocation randomness were analyzed for a variety of biased coin probabilities (see Table 3). The P-values for covariate imbalance tests were obtained from a list compiling all Chi-squared tests and t tests in each simulation study and reflects overall balance, rather than imbalance on certain covariates. The results indicate three significant findings. First, minimization was inferior when compared to MSB at controlling covariate imbalance for fixed biased coin probabilities of .55, .6, and .65. Second, the overwhelming majority of minimization-based treatment assignments were made using a biased coin favoring balance between treatment groups. Even for a biased coin probability of 1.0, less than 20% of the treatment assignments were purely random, as opposed to 82.3% of the assignments for MSB (see Figure 1). Third, in minimization, the correct guess probability was high, especially as the biased coin probability increased. On the other hand, the correct guess probability for MSB increased as the biased coin probability increased but at a fraction of the rate (see Figure 2). When five covariates were balanced using biased coin probabilities of .7, .8, .9, and 1.0, the fifth percentile of the P-values from balance tests for minimization was above both the corresponding value from the MSB method and the imbalance threshold of .3.

TABLE 3.

Results from simulations related to allocation randomness when biased coin probability is varied

Minimization
Biased coin probability	ξ	0.5	0.55	0.6	0.65	0.7	0.8	0.9	1.0
Proportion of pure random assignments	α	1.00	.095	.119	.137	.153	.172	.184	.192
Correct guess probabilitya	G	.500	.545	.588	.629	.669	.748	.826	.904
Balance test P-value fifth percentile	P0.05	.053	.121	.207	.280	.352	.396	.432	.434
Minimal sufficient balance
Biased coin probability	ξ	0.5	0.55	0.6	0.65	0.7	0.8	0.9	1.0
Proportion of pure random assignments	α	1.00	.421	.534	.613	.667	.736	.789	.823
Correct guess probabilitya	G	.500	.529	.547	.558	.567	.579	.584	.589
Balance test P-value fifth percentile	P0.05	.056	.169	.255	.288	.297	.313	.319	.318

^aG = (0.5)α + (1−α)ξ.

FIGURE 1.

Biased coin probability vs proportion of completely random treatment assignments for MSB and minimization

FIGURE 2.

Biased coin probability vs correct guess probability for MSB and minimization

4.5 |. Impact of P-value threshold in MSB

When implementing the MSB method, larger P-value cutoffs resulted in improved covariate balance and fewer completely random treatment assignments (see Table 4). The proportion of pure random assignments under MSB was much greater than minimization in all cases, including the P-value cutoff of .4 (see Table 3).

TABLE 4.

Results from simulations varying the P-value cutoff in MSB

P-Value cutoff	P	.1	.2	.3	.4
Proportion of pure random assignments	α	.839	.755	.675	.601
Correct guess probability	G	.524	.537	.549	.560
Balance test P-value fifth percentile	P0.05	.129	.208	.292	.367

5 |. DISCUSSION

The results of this study indicate that MSB was able to control covariate balance across treatment arms in equal allocation sequential clinical trials. In all cases where the biased coin probability was equal or less than .65, overall covariate balance is better in MSB than in minimization. However, superior balance existed for minimization for biased coin probabilities of .7 and above. In all cases where minimization showed superior overall balance for a fixed biased coin probability, the fifth percentile of the P-values for the imbalance tests was near or above the prespecified threshold for MSB of .3. As a result, it should be interpreted that both methods were effective at controlling imbalance using biased coin probabilities of .7, .8, .9, and 1.0. This is because imbalances P-values below .3 were rare (ie, present less than 5% of the time) using both methods. In addition to MSB having superior or equal balance when compared to minimization for a fixed biased coin probability, the loss of efficiency when balancing 11 covariates compared to 5 was less significant when using MSB than when using minimization. Thus, MSB is better able to control imbalance on many covariates than is minimization.

In this study, MSB consistently performed well at balancing continuous covariates. Minimization balanced categorical variables to a much greater degree than continuous variables when biased coin probability was fixed, resulting in better balance of categorical covariates across treatment arms. It should be noted that Kuhn et al²⁴ have shown that when deterministic minimization is used with four categorical covariates, minimization asymptotically controlled serious imbalances better than the MSB method. However, minimization is commonly used with continuous covariates^11,34–36 and deterministic treatment assignments are discouraged in practice. In this study, minimization did not perform well at balancing continuous covariates. The reason for this is because, although covariates are balanced according to the categorical form of the variables, balance is measured at the continuous scale.¹⁰ Polychotomization required in the minimization method contributes to a major loss of efficiency when controlling imbalance in continuous variables. In this study, continuous variables were polychotomized at quartiles into four approximately equally-sized groups. Polychotomization of continuous variables is generally a subjective process and is not applied consistently in clinical trials.¹¹ Some clinicians use preliminary data to inform the process of polychotimization, yet often dichotimization or some other form of polychotomization is performed with little consistency across trials. In this study, authors considered using two and four categories with different cut points for dichotomization and results were generally similar. However, some variables showed better balance with two categories and others showed better balance with four categories. On average, however, balance was somewhat better with four categories and these results are presented in Tables 1–3.

MSB is less sensitive to selection bias than minimization because the correct guess probability is extremely low for all biased coin probabilities considered. Thus, if clinicians become unblinded, major selection bias can take place in minimization-based trials. Pure randomization is an important aspect of a clinical trial which can protect against imbalance, especially on unknown or unobserved prognostic variables. However, this study found that when minimization is used for assigning subjects to treatments, less than 20% of treatment assignments are completely random.

The simulation study on MSB is a guide for researchers on how to manage the tradeoff between allocation randomness and covariate balance when implementing MSB (see Table 4). To reduce the potential for selection bias, the P-value cutoff should be made as small as possible (assuming P-values are used to make decisions in the MSB method) to avoid potential for serious imbalances. The P-value cutoff of 0.3 was used in this study for all simulations, unless otherwise specified. This was done to keep all or almost all covariate imbalance P-values above 0.2 because these are considered minor imbalances.¹⁴ However, even if a P-value cutoff were set to 0.4, the allocation randomness would be much greater for MSB than for minimization.

The MSB method of covariate balance coincides with the philosophical reason for balancing covariates: completely random design is ideal, but not feasible for small clinical trials and interim analyses. Thus, MSB does not interfere with randomization unless there is a serious imbalance. In addition, MSB is not recommended over completely random design for large sample sizes (greater than 500 subjects) when random occurrence of serious imbalances is unlikely. However for interim analyses of large clinical trials, MSB can help guarantee that a decision to stop for futility or efficacy is based upon strong inference.

With small or medium sample sizes, MSB proves to be a covariate balance method that is more effective at balancing covariates and more effective at maintaining allocation randomness, when compared to minimization. Minimization seeks to balance even the most minimal differences between treatment groups and does so by minimizing the overall imbalance, becoming less effective when the number of covariates increases. In addition, minimization does not have the flexibility that is present in MSB. Although the MSB method proposed by Zhao et al¹⁴ is relatively new, it has been used in clinical trials such as in the ESCAPE trial⁴⁰ and in the SHINE trial¹⁶ and may be used more widely as research is conducted on the limitations of stratified permuted blocks and minimization.

Permutation tests suggest that type I error rates were preserved in general, for each of the randomization methods considered. However, further exploration to examine the effect that each method might additionally have on the power and bias in estimates of treatment effects is warranted. This exploration is especially important, because sample size and power calculations typically assume a completely random design, which may not be appropriate if minimization or MSB methods are used for randomization. This is currently being explored and the results will be presented in subsequent publications. Additional investigations will extend these results to the case of unequal allocations. In doing so, a more global comparison of minimization and MSB may be observed in order extend the results found to the analysis phase of a trial and to cases of unequal allocation to further showcase properties of MSB as they compare to other commonly used methods of covariate-adaptive randomization.