Understanding Superiority, Noninferiority, and Equivalence for Clinical Trials

Abstract

Randomized control trialsare the gold standard for testing the efficacy of new interventions. Historically, superiority trials were methods of choice as reference (standard) interventions were not established for many disease conditions. However currently, reference interventions are available for most of adverse conditions. Despite this, many investigators are using superiority trials in comparison to more suitable noninferiority and equivalence trials. The application of noninferiority and equivalence trials is on the rise, but by and large, these trials are poorly understood, ill-conceived, inappropriately analyzed, and reported and misinterpreted.

Introduction

The efficacy of a drug in randomized control trials (RCTs) is established using null hypothesis significance testing (NHST) approach.[1,2] An insightful clinical trial meticulously documents the specific objectives, hypotheses, data analysis, and reporting plans. Traditionally, the investigators were investigating the superiority of the intervention group with a control group in the absence of reference interventions. Therefore, the classical approach was to frame two-sided (there is a significant difference in the outcome variable between two groups) alternative hypothesis. The Consolidated standard of Reporting (CONSORT) for RCTs recommend two-sided P value and confidence interval approach to declare the statistical significance.[3] After rejecting the null hypothesis, investigators used to compare the average score of the outcome variable between the intervention group (experimental group) and the control group to declare superiority. Recently, many authors are comparing novel interventions (NI) as compared to reference interventions (RI). The competing intervention may not significantly differ in terms of their efficacy. Although, there maybe various other reasons such as safety, low cost, ease of administration, and lesser side effects of a novel intervention.[4,5] Therefore, the traditional approach of superiority trial do not work in these types of situations. The noninferiority and equivalence trials are better methods of investigations for comparing NI against standard treatment. The primary objectives of noninferiority and equivalence trials are to demonstrate that NI is better or not worse than active control (standard intervention or reference intervention) beyond clinically significant margins.[5,6] By contrast, noninferiority trials investigate NI (e.g., new drug or treatment claimed to be better than existing standard drug by the company in terms of ease of administration, etc.) is not worse than active control in terms of either or both safety and efficacy beyond a specified margin of clinical significance. However, NI may be better than active control. Equivalence trial investigates novel intervention (e.g., generic drugs claimed by the govt. to have similar efficacy and safety but lower cost as compared to company supplied drug) is not better and worse than active control beyond the prespecified margin of clinical significance. Thus, null and alternative hypothesis and type-I and type-II error will change as per the objectives of the trial. There will also be certain modifications in sample size and statistical analysis plan. The classical approaches of the null hypothesis, sample size, statistical analysis using the P value are not justifiable for non-inferiority and equivalence trials. Therefore, the framing of appropriate hypotheses is foundations for the application of appropriate superiority, noninferiority, and equivalence trials. Further, it facilitates sample size, statistical analysis, reporting, and conclusions. Initially, we attempt to disentangle the various intricacies of noninferiority and equivalence trial along with familiar superiority trial. Finally, we proposed a brief table of important characteristics to plan and analyses these types of trials.

Clinically meaningful difference (Δ)

The concept of statistical significance and clinical importance is at the heart of clinical research. The clinically relevant difference should always take precedence over statistical significance. A change in score of few points (e.g., 70-65) in the severity of alopecia assessment tool (SALT; range 0-100) among alopecia patients may be statistically significant but may not be clinically relevant.[7] Typically, a clinically meaningful difference is determined clinically and statistically after consulting literature, personal experience and discussion with colleagues. It is denoted with symbol Δ and is usually challenging to select. A very small Δ may lead to rejection of a promising drug, whereas large Δ may lead to a selection of potentially inferior drug. Generally, superiority trials justify Δ after analysis as compared to equivalence and noninferiority trials. Whereas, a prerequisite knowledge about Δ is vital to frame hypotheses for equivalence and noninferiority trials.[8] The conclusion about equivalence and noninferiority also depends on Δ. It is compelling to specify Δ after data inspection, but it may give rise to bias.[9] Therefore, it is essential to fix Δ with suitable reasons in advance while finalizing research protocol. A conservative rule is to set a clinically significant difference (Δ) of 50% from the value of lower confidence interval in a superiority trial.[6] The FDA recommended the Δ value of 20, 15, 10, and 5% for the anti-infective drugs with the efficacy of 50-80, 80-90, 90-95, and >95%, respectively, measured as the binary outcome.[10]

Hypothesis and error

An initial step in almost all the studies is to formulate the hypothesis. A superiority trial investigates whether NIis better than active control (AC) or placebo by a specified margin (Δ). The null and alternative hypothesis for superiority trials are H₀:μ_NI- μ_AC≤Δ and H₁:μ_NT- μ_AC>Δ, respectively. However, in practice, a superiority trial is a two-step NHST process. Therefore, as per NHST, null and alternative hypothesis are formulated as H₀:μ_NT- μ_AC= 0 and H₁:μ_NT- μ_AC≠ 0. After rejecting the null hypothesis,^th e adequacy of Δ is determined while reporting results. The authors can read an excellent article written by Farrugia et al. on formulating the research question, hypotheses and objectives.[11] The equivalence trial investigates whether NI is equally good as compared to AC. The active control is the standard intervention for noninferiority and equivalence trials as compared to placebo or non-standard intervention for superiority trial. The equally good margin ranges from -Δ to +Δ and known as tolerance range for equivalence studies. The noninferiority trials examine whether NI is not noninferior to AC. The NI will be noninferior to AC (standard intervention) if the noninferiority margin is to right of -Δ. The type-I error is critical for conclusions. Type-I error is falsely rejecting a null hypothesis (i.e., falsely declaring statistical significance). The interpretation is inherently confusing to investigators in general for equivalence and noninferiority trials. The objectives of null and alternative hypotheses for superiority (rejection of null) and equivalence trials (rejection of difference) are opposite to each other. Whereas alternative hypothesis for noninferiority trial is one-tailed as compared to two-tailed for superiority trials. Researchers can refer Table 1 for a lucid explanation of hypotheses and errors for the different type of trials.

Table 1.

The hypotheses and errors for different type of clinical trials

Type of trial	Null hypothesis	Alternative hypothesis	Type-I error	Type-II error
Superiority	There is no significant difference between novel intervention and Placebo	There is a significant difference between the novel intervention and placebo	Erroneously concluding superiority of novel intervention when there is no superiority	Not concluding a superiority when there is a superiority
Noninferiority	The novel intervention is not noninferior to reference intervention	The new intervention is noninferior to reference intervention	Erroneously concluding noninferiority of novel intervention when there is no noninferiority	Not concluding noninferiority when there is noninferiority
Equivalence	The novel intervention is not equivalent to reference intervention	The novel intervention is equivalent to reference intervention	Erroneously concluding equivalence of interventions when there is no equivalence	Not concluding equivalence when there is equivalence

Sample size

The optimum sample size is one of the crucial requirements to conduct systematic and objective research. It is unethical and unscientific to conduct underpowered studies. Typically, there are four fundamental requirements, such as variance, confidence interval (CI), power, and effect size to calculate the sample size. However, it is essential to prespecify with an appropriate explanation for noninferiority and equivalence trials. Table 2 gives the formula for calculating sample size for continuous and binary outcome variables in different types of clinical trials. The researcher interested in detailed discussion regarding sample size for many other types of designs can consult “Sample Size Calculations in Clinical Research” by Chow et al.[10] The total sample size (n) is sum (n₁+n₂) of sample sizes in each arm. The value ofk = 0.5, 1 and 2 represent 1:2, 1:1, and 2:1 participants in treatment and control group, respectively. Typically, the value of is taken as zero for superiority trials as the null hypothesis is usually framed as a hypothesis of no difference. In general, sample size to conduct noninferiority and equivalence studies are higher as compared to superiority trials.[9] Let us assume the effectiveness of achieving an intended endpoint (say PASI75) with a placebo and a drug under investigation is 25% and 60%, respectively. Normally, = 0 for superiority trials. The and values at 95% CI and 80% power are 1.96 and 0.84, respectively. The values of P₁ = 0.25 and P₂ = 0.60 and d = 0.25–0.60 = -0.35. Assuming, investigator need equal number of cases to controls (K = 1). The application of sample size formula for superiority trial from Table 2 give a sample of approximately 29 individuals to be recruited in each group. Similarly, assume the effectiveness of achieving PASI75 for noninferiority trial with a standard drug and novel interventions are 55% and 60%, respectively. The and values at 95% CI and 80% power are 1.64 and 0.84, respectively. The values of P₁ = 0.55 and P₂ = 0.60 and d = 0.55–0.60 = -0.05. A noninferiority margin of 10% (= -0.10) is considered meaningful for the outcome. Assuming, investigator need equal number of cases to controls (K = 1). The application of sample size formula for noninferiority trial from Table 2 give a sample of approximately 134 individuals to be recruited in each group. Similarly, assuming researcher is interested in equivalence trial for the measures given for noninferiority trials. All the values except values at 95% CI and 80% power are 1.96 and 1.28 will remain the same. The application of sample size formula for equivalence trial from Table 2 give a sample of approximately 2047 individuals to be recruited in each group.

Table 2.

Sample size formulae for different type of trials for continuous and binary outcome variable

	Continuous outcome variable	Binary outcome variable
Superiority trial
Noninferiority trial
Equivalence Trial

z₁-α = 1.64, 1.96, and 2.58 for 90%, 95% and 99% for 2-sided confidence interval respectively, z_β =0.84 and 1.28 for 80% and 90% power respectively, σ²=Expected variability (obtained from previous studies). P₁ = Proportion of participants with outcome in the first group, P₂=Proportion of participants with outcome in the second group. k=Allocation ratio between treatment and control group, d =Effect Size, ∆=Clinically meaningful difference

Statistical analysis

The CONSORT guidelines recommend using the P value and CI approach for declaring the superiority of one intervention over others.[3] However, many studies still report results using the P value approach. The superiority using CI can be declared if the CIdoes not include 0 (1 for odds and risk ratio). When it comes to making inference about non-inferiority and equivalence trials, CI approach is recommended in comparison to the P value approach. The intention to treat (ITT) and per-protocol analysis have advantages and disadvantages. ITT is the preferred analytic approach for reporting results from superiority trials in comparison to both ITT and per-protocol analysis for equivalence and noninferiority trials.[6,12] Usually, both lead to the same conclusion. However, in case of disagreement, a judicious and careful approach after consulting literature needs to be adopted. The clinically significant difference (Δ) plays an essential role in declaring non-inferiority and equivalence. The CI to declare equivalence lies between –Δ and +Δ. For the noninferiority trial, novel intervention needs to be similar or better than reference intervention. Therefore, the CI to declare non-inferiority lies on the right side of –Δ. Figures 1 and 2 display the confidence interval approach to declare significance for positive (higher the better) and negative (lower the better) outcome variable, respectively, for various trials.

Figure 1.

Interpretation of results from confidence intervals with different type of trials for positive outcome

Figure 2.

Interpretation of results from confidence intervals with different type of trials for negative outcome

Contextualizing

Scenario

Understanding the choice of trial design based on the evolution of first-line systemic therapies in severe psoriasis from methotrexate to biologics and biosimilars.

The relevance of understanding the trial design for superiority, inferiority, or equivalence can be understood in context to the gradual emergence of biologic response modifiers in the management of severe psoriasis. At the turn of the century, there were limited data on the clinical efficacy and adverse effect profile as targeted therapies were emerging for treating severe psoriasis. The early randomized controlled trials were placebo-controlled.[13] It leads to establishing of methotrexate as a standard drug. Subsequently, a three-arm noninferiority trial popularly known as CHAMPION trial compared adalimumab (NI) with the gold standard drug methotrexate,[14] and a placebo. Despite being designed as a non-inferiority trial, the CHAMPION trial declared the superiority of adalimumab as compared to methotrexate. The Committee for Proprietary Medicinal Products (CPMP) have discussed conditions under which results of noninferiority trial can be reported as a superiority trial.[9]

As mentioned above, placebo-controlled superiority trials were the first steps for new drugs to show its potential benefits. The recommendations are for noninferiority designs to compare the new drug (adalimumab in this case) relative to an established one (methotrexate here). The CHAMPION trial was designed as a double-blind, double-dummy placebo-controlled study with an adequate sample size of 90% power to detect noninferiority of adalimumab relative to methotrexate (based on the primary outcome of achieving PASI75- assuming effectiveness of adalimumab to be 62% and of methotrexate to be 60%, and placebo 4%).[14] Despite few limitations or criticisms of the trial, CHAMPION trial proved to be a vital trial in turning the tide toward the more widespread use of biologics in comparison to the traditional systemic agents in managing moderate to severe psoriasis. Recently, many biosimilars came into the market after the expiry of patent for adalimumab (Humira) in both the United States and Europe. Hercogova et al. compared the efficacy, safety, and immunogenicity of the biosimilar MSB11022 (a new agent here) with reference adalimumab (now an established agent).[15] Before the clinical trial, the biosimilar was shown to have structural, functional, and pharmacokinetic equivalence to adalimumab. The researchers can consult Table 3 as a ready reference to decide about the selection of one trial over another.

Table 3.

Summary of important characteristics of difference between different type of trials

	Type of trial
Characteristics	Superiority trial	Noninferiority trial	Equivalence trial
Condition for application	Comparing novel intervention with non-standardized intervention	Comparing novel intervention with standard intervention	Comparing novel intervention (e.g., generic drug) with standard intervention
Control	Placebo or non-standard Intervention	Standard intervention	Standard intervention
Intervention	Novel intervention	Novel intervention	Novel intervention
Hypothesis	Two-tailed	One-tailed	Two-tailed
Key to hypothesis formation	Effect size (d)	Effect size (d) and clinically meaningful difference (∆)	Effect size (d) and clinically meaningful difference (∆)
Statistical Significance range	µ1-µ0≠0	µ1-µ0> -∆	-∆µ1-µ0≤∆
Analysis recommendation	ITT	ITT and per protocol analysis	ITT and per protocol analysis
Reporting	P-value and CI	CI	CI

CI - Confidence intervals; ITT - Intention to treat

Conclusion

By default, superiority trials are preferred by researchers. However, the usage and reporting of noninferiority and equivalence trials are increasing. These trials are more complex and challenging to understand and interpret as compared to superiority trials. The researchers need to better equip themselves with the intricacies and subtle differences between various trials to keep themselves abreast with the latest developments. Careful attention will help researchers to make an informed decision about the claimed safety and efficacy of interventions.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.