Head-to-head trials of systemic psoriasis therapies: A systematic review of study design and maximum acceptable treatment differences.

Abstract

There is increasing use of head-to-head clinical trials in dermatology when establishing the efficacy of a new treatment. Active comparator trials (ACTs) can be classified into three distinct study trial designs: non-inferiority, equivalence, and superiority. A better understanding of the statistical parameters, such as acceptable treatment differences (also known as the margin or delta), is necessary to properly design and interpret findings of active comparator trials (ACTs) in the field of dermatology. Therefore, the objective of this study is to summarize the maximum acceptable treatment differences in clinical trials that examine the efficacy of an oral or biologic psoriasis therapy with an active comparator. We conducted a systematic search using MEDLINE, Scopus, EMBASE, Cochrane Central Register of Controlled Trials, LILACS, Web of Science, and ClinicalTrials.gov from inception to August 31, 2017. All ACTs with adult participants that had a primary outcome of the Psoriasis Area and Severity Index score were included. Bibliographies of articles were further reviewed. Two investigators independently assessed for article inclusion and separately completed data extraction of pre-defined data points. When there was a disagreement, a third investigator was consulted. Of the 49 ACTs included, there were 13 superiority, 8 non-inferiority, and 7 equivalence trials. Another 21 studies had inadequate information for classification. All of the non-inferiority trials reported the margin, 1 of the superiority, and 6 of the equivalence trials stated the treatment difference explicitly. For superiority trials, acceptable treatment differences ranged from 14% to 20%. The non-inferiority studies reported lower bound margins ranging from −20% to −10%. The equivalence trials reported upper and lower bound margins ranging from ±12.5% to ±18%. The results demonstrate the need for harmonization in the conduct of dermatological clinical trials and in the approaches of reporting research parameters.

Introduction

Clinical trials may include the use of placebo (PBO), active treatment (an approved therapy, standard care, or the best available intervention), or no control group.¹ Active-controlled clinical trials (ACTs) are increasingly the benchmark in many fields of medicine when establishing the efficacy of a new treatment.^2,3 Even if ACTs do not achieve the desired primary outcomes or prove the hypothesis, negative results still provide highly valuable information⁴ by identifying sub-populations who may stand to gain the most benefits and ascertain those in whom the drug should be avoided. Prior to the 2000’s, ACTs were quite uncommon for systemic agents in moderate to severe psoriasis. For example, a systematic review of dermatological studies from 1977–2000 found only 31 (12.5%) randomized controlled trials (RCTs) comparing treatment modalities in different therapeutic classes.⁵ Although placebo-controlled trials often precede active-controlled trials in psoriasis research, head-to-head comparisons for new therapies are becoming the rule rather than the exception.⁶ Preference for ACTs may be attributed to the pragmatic nature of such trials as they assess the comparative values of different treatments as opposed to choosing to treat or not (i.e., PBO). However, head-to-head clinical trials present unique challenges in the design and the interpretation of results.

ACT methodology can be classified into different types of comparisons: non-inferiority trials (which aims to demonstrate a minimum level of efficacy in the investigational intervention vs. the comparator by a pre-specified amount); equivalence trials (often employed in biosimilar studies to evaluate whether the investigational intervention is comparable enough, or not “too different,” to the originator drug); and superiority trials⁷ (the most convincing way to determine whether the investigational intervention is more efficacious than the comparator) (Fig. 1).^8,9 It is critical to understand the type of comparison as the design directly impacts the conclusions that can be drawn from the results. Particularly, it is crucial that the margins or delta of comparison, also known as the clinically acceptable maximum difference between two treatments, be well-defined and justified as this difference directly influences sample size estimates as well as study conclusions.

Figure 1.

Statistical differences between superiority, equivalence, and non-inferiority clinical trials.

Figure 1a. Superiority Clinical Trial Principles illustrated by different scenarios (1–4), where the effects of an active control and experimental treatment are within the confidence intervals (along the x-axis).

1) Experimental is superior to active control. The CI does not include zero treatment difference.

2) Experimental is superior to active control but it has a wider CI than in (1), meaning that the estimate of the difference between treatments is less precise (due to more variance or a smaller sample size).

3) There is no difference between treatments.

4) The entire CI lies below the line of Δ0, favoring active control.

We recently conducted a systematic review of active-controlled trials of biosimilars and found that 5 out of the 9 studies did not report adequate information to ascertain key aspects of study design, which led us to conduct a larger systematic review inclusive of all systemic and biologic ACTs in psoriasis.¹⁰ Given the increasing use of active controls, development of new investigative therapies, and implications for evidence-based practice, we sought to determine the range of statistical parameters for active-controlled systemic and biologic therapeutic trials in psoriasis. The objective of this study is to summarize the different types of ACTs (superiority, NI, or equivalence trials) and to synthesize current evidence to inform future comparative trials in psoriasis.

Methods

We conducted a systematic search based on the Cochrane Handbook for Systematic Reviews of Interventions guidelines using MEDLINE, Scopus, EMBASE, Cochrane Central Register of Controlled Trials, LILACS, Web of Science, and ClinicalTrials.gov from inception to August 31, 2017¹¹ to identify eligible articles using relevant keywords and Medical Subject Headings with the assistance of a professional librarian (Supplementary Table 1). To avoid possible publication bias and identify ongoing or unpublished trials, we also searched clinicaltrialresults.org, World Health Organization International Clinical Trials Registry Platform and Food and Drug Administration websites on September 7, 2017. To obtain data from unpublished studies and abstracts, we contacted corresponding authors or industry sponsors. Supplementary data were retrieved if the full-text publications did not contain data points of interest. Our inclusion criteria (Supplementary Table 2) included adult patients (≥18 years old) with plaque psoriasis (PsO) randomized to receive monotherapy of an experimental intervention or active control. Our outcome of interest was the Psoriasis Area and Severity Index (PASI) score, as this is the measure typically used in clinical trials to define successful treatment,¹² measured at week 12 or later.

Study Selection

Study selection was carried out according to the PRISMA statement (Fig.2).¹³ Two reviewers (MTW and JA) independently reviewed the titles and abstracts of all studies for inclusion and exclusion criteria which had been downloaded from the search into EndNote X7.7.1 (Thompson Reuters). EndNote X7.7.1 and STATA 13.1 (College Station, Texas, USA) were used to identify duplicates. Titles and abstracts were screened for inclusions, followed by full-text retrieval if abstracts were insufficient to determine whether studies met inclusion and exclusion criteria. In cases of disagreements, the two reviewers (MTW and JA) compared findings and reached consensus about eligibility and on classification. If consensus was not reached, a third reviewer (EDD) was consulted. Single-center studies were excluded as small sample sizes typically yield studies that are inadequately powered. Although our search criteria included phototherapy, we excluded phototherapies after the database search in addition to topical treatments as PsO RCTs generally examine oral and biologic therapies.

Figure 1b.

Non-inferiority Clinical Trial Principles illustrated by different scenarios (1–6), where the effects of an active control and experimental treatment are within the confidence intervals (along the x-axis).

1) Experimental non-inferior to control but not significantly different from active control.

2) Experimental is non-inferior to control but is significantly different from control.

3) Experimental is somewhat inferior to control. Experimental is not inferior because the CI does not include the non-inferior margin or the line of Δ0.

4) Inconclusive because the CI includes values on both sides of the non-inferiority margin and the line of no treatment difference (Δ0).

5) Experimental is inferior to control because it includes the non-inferiority margin.

6) Experimental is inferior to control and is more certain of its inferiority as compared to scenario (5) because it does not include the non-inferiority margin.

Data Extraction and Management

Two authors (MTW and JA) independently extracted pre-determined data from all 49 RCTs that met inclusion criteria. RCTs were classified into superiority, NI, or equivalent trial design based on the language used in the publications and the reported statistical parameters between the experimental and active control groups. ACTs were deemed as “other head-to-head comparisons” if the authors could not retrieve adequate information to classify the RCTs as one of the three aforementioned study designs. For the purpose of this manuscript, ACTs were classified according to the type of comparison of the investigative drug versus the active control (i.e., a study was considered as a NI ACT if any of the hypotheses tested non-inferiority of an investigative intervention versus active control, even if it also explored superiority to placebo). For studies with multiple hypotheses tested, ACTs were classified according to whichever hypothesis was tested first sequentially. We abstracted the following data: study design, sample size, power, confidence intervals, margin or delta, primary outcome, and response rates (Tables 1–4). From the extracted data, we summarized ranges for continuous data and calculated proportions and percentages for categorical data. Studies with missing values were excluded from specific analyses. Statistical analyses were performed using STATA 13.1.

Table 1.

Characteristics of the Superiority Randomized, Controlled Trials with an Active Comparator.

Source	Sponsor (Completion Year)	Title	Experimental	Active Comparator	PBO	Sample Size (Experiment/Comparator/PBO) A priori/Observed	Primary Outcome	Week Measured	Response Rate A priori/Observed	Delta (%)a	CI (%)	Power	Study Qualityb
Table1A. Superiority Trials with complete power analyses data.
NCT0067973132	AbbVie (2009)	A 52-week Trial Comparing Briakinumab with Methotrexate in Patients with Psoriasis	Briakinumab	MTX	-	250 (1:1) 317 (154/163)	PASI75	24	70% (Briakinumab) vs. 50% (MTX)	20	95	0.9	1
NCT0045458433	Centocor (2009)	ACCEPT	UST – 45mg & 90mg	ETA	-	850 (3:5:5)c 903 (209/347/347)	PASI75	12	65% (UST 90mg) vs. 50% (ETA)	15	95	>0.97	1
									64% (UST 45mg) vs. 50% (ETA)	14	95	0.87	1
NCT0071058034	AbbVie (2013)	Study Comparing the efficacy and safety of ABT-874 to Etanercept and Placebo in subjects with Moderate to Severe Chronic Plaque Psoriasis	Briakinumab	ETA	PBO	350 (2:2:1) 350 (139/139/72)	PASI75	12	70% (Briakinumab) vs. 50% (ETA) vs. 4% (PBO)d	20	95	0.9	1
NCT0069196435	AbbVie (2013)	Study Comparing the efficacy and safety of ABT-874 to Etanercept and Placebo in subjects with Moderate to Severe Chronic Plaque Psoriasis	Briakinumab	ETA	PBO	350 (2:2:1) 347 (138/141/68)	PASI75	12	70% (Briakinumab) vs. 50% (ETA) vs. 4% (PBO)d	20	95	0.9	1
NCT0205448116	Boehringer Ingelheim (2016)	BI 655066 Dose Ranging in Psoriasis, Active Comparator Ustekinumab	BI655066 (RIS) – 18mg & 90mg & 180mg	UST	-	160 (2 pooled:1) 166 (43/41/42/40)	PASI90	12	70% (pooled 90mg and 180mg RIS) vs. 45% (UST)	25	90	>0.8	1
NCT0172975429	Merck (2019)	reSURFACE2	Tildrakizumab – 200mg & 100mg	ETA	PBO	1050 (2:2:1:2)	PASI75	12	73% (TIL) vs. 56% (ETA)	17	95	>0.98	1
Table1B. Superiority Trials with incomplete power analyses data.
NCT0170860336	Amgen (2014)	AMAGINE-2	BRO - 140mg & 210mg	UST	PBO	1800 (2:2:1:1) 1831 (612/610/300/309)	PASI100	12	26% BRO 140mg (p=.08) & 44% BRO 210mg (p<.001) vs. 22% UST	NR	95	≥0.9	1
NCT0170862936		AMAGINE-3				1800 (2:2:1:1) 1881 (624/629/313/315)	PASI100	12	27% BRO 140mg (p=.007) & 37% BRO 210mg (p<.001) vs. 19% UST	NR	95	≥0.9	1
NCT0207498237	Novartis (2017)	CLEAR	AIN457 (SEC)	UST	-	640 (1:1) 676 (338/338)	PASI90	16	71% (SEC) vs. 51% (UST)	20	NR	>0.99	1
NCT0220723138	Janssen (2020)	VOYAGE 1	Guselkumab 100mg	ADA	PBO	750 (2:2:1) 837	PASI90	24	73.3% (Guselkumab) vs. 49.7% (ADA) vs. 2.9% (PBO)e	NR	95	NR	1
NCT0220724439	Janssen (2020)	VOYAGE 2	Guselkumab 100mg	ADA	PBO	992 (2:1:1)	PASI90	24	70.0% (Guselkumab) vs. 46.8% (ADA) vs. 2.4% (PBO)e	NR	95	NR	UA
Table1C. Unpublished Superiority Trials
NCT0282660340	Novartis (2018)	CLARITY	SEC	UST	-	1109	PASI90	12	UA	UA	UA	UA	UA
NCT0316825641	Can-Fite BioPharma (2019)	CF101 Therapy in Patients With Moderate-to-severe Plaque Psoriasis	CF101 – 2mg & 3mg	Apremilast	PBO	407 (3:3:3:2)	PASI75	16	UA	UA	UA	UA	UA

The italicized numbers are the observed data, instead of the expected response rates and sample size required for power analyses decided a priori.

Abbreviations: Bi-weekly (BIW); Briakinumab (BRI) brodalumab (BRO); every 2 weeks (Q2W); every 4 weeks (Q4W); every 8 weeks (Q8W); every 12 weeks (Q12W); Adalimumab (ADA); Confidence Interval (CI); Etanercept (ETA); Infliximab (INF); methotrexate (MTX); not reported (NR); placebo (PBO); Risankizumab (RIS); Secukinumab (SEC); subcutaneous (SC); unavailable data point (UA); ustekinumab (UST).

^aThe hypothesized true magnitude of difference between the arms (or effect size) was calculated according to reported statistical parameters from the manuscript that were decided a priori. The reported delta is based on the experimental vs. active comparator.

^bUsing a modified rating of individual studies from the Oxford Centre for Evidence-based medicine,⁴² 1 indicates a properly powered and conducted randomized clinical trial.

^cPower to detect treatment difference at an alpha level of 0.05 between 90mg UST and ETA was 0.97 and between 45mg and ETA was 0.87.

^dThese reported power calculations were based on PGA 0/1 (co-primary endpoint), not PASI75.

^ePower calculations based on experimental vs. placebo had >0.99 power. Secondary statistical analyses included the experimental vs. active control.

Table 4.

Head-to-head research trials that could not be classified as any of the aforementioned three statistical designs.

Source	Sponsor/Author (Completion Year)	Title	Study Arms	Sample Size (Experimental/Active Comparator/PBO) A Priori/Observed	Primary Outcome	Week Measured	Response Rate of Primary Outcome	CI (%)	Delta / Margin	Study Power	Study Qualitya
Table 4A. Published
N/A	Flystrom et al17 (2005)	Methotrexate vs. ciclosporin in psoriasis: effectiveness, quality of life and safety. A randomized controlled trial	MTX vs. Ciclosporin	70 (35/35)	mean change in PASI	12	58% (MTX) vs. 72% (ciclosporin)	95	NR	0.9	1
NCT0025164158	Merck (2008)	RESTORE1	INF vs. MTX	868 (653/215) 800 (3:1)	PASI75	16	78% (INF) vs. 42% (MTX)	95	NR	NR	1
N/A	Noor et al18 (2011)	Efficacy and safety of MTX vs. acitretin in chronic plaque psoriasis	MTX vs. Acetretin	142 (71/71)	PASI75	24	53.5% (MTX) vs 25.3% (Acetretin)	NR	NR	NR	1
NTR155919	De Vries et al (2012)	PIECE	INF vs. ETA	48 (25/23) 120	PASI75	12	75% (INF) vs 50% (ETA)	95	NR	0.8	1
Table 4B. Unpublished
NCT0169029959	Celgene (Apr 2016)	LIBERATEb	Apremilast (30mg BID) vs. ETA (50mg SC weekly) vs. PBO	250 (1:1:1)	PASI75 & PASI 50	16	UA	UA	UA	UA	UA
NCT0247408260	Novartis (June 2016)	PRIME	SEC vs. Fumaric Acid	200	PASI75	24	UA	UA	UA	UA	UA
NCT0248922761	Coherus Biosciences (Mar 2017)	Comparison of CHS-1420 Versus Humira in Subjects With Chronic Plaque Psoriasis	CHS-1420 (ADA biosimilar) vs. ADA (HUMIRA®)	545	PASI75	12	UA	UA	UA	UA	UA
NCT0258134562	Momenta (Apr 2017)	Phase 3 Study of M923 and Humira® in Subjects With Chronic Plaque-type Psoriasis	M923 vs. ADA (HUMIRA®) vs. M923 & ADA	827	PASI75	16	UA	UA	UA	UA	UA
NCT0268437063	AbbVie (Aug 2017)	BI 655066 (RIS) Compared to Placebo and Active Comparator (UST) in Patients With Moderate to Severe Chronic Plaque Psoriasis	ABBV-066/BI 655066 (RIS) vs. UST vs. PBO	500	PASI90	16	UA	UA	UA	UA	UA
NCT0269452364	AbbVie (Aug 2017)	BI 655066/ABBV-066 (RIS) Compared to Active Comparator (ADA) in Patients With Moderate to Severe Chronic Plaque Psoriasis	ABBV-066/BI 655066 (RIS) vs. ADA vs. PBO	605	PASI90	16	UA	UA	UA	UA	UA
NCT0268435765	AbbVie (Sept 2017)	BI 655066 Compared to PBO & Active Comparator (UST) in Patients With Moderate to Severe Chronic Plaque Psoriasis	ABBV-066/BI 655066 (RIS) vs. UST vs. PBO	500	PASI90	16	UA	UA	UA	UA	UA
NCT0263480166	Eli Lilly (Nov 2017)	A Study of Ixekizumab (LY2439821) in Participants With Moderate-to-Severe Plaque Psoriasis Naive to Systemic Treatment	Izekizumab vs. Fumaric Acid vs. MTX	162	PASI75	24	UA	UA	UA	UA	UA
NCT0285096567	Boehringer Ingelheim (Dec 2017)	Efficacy, Safety and Immunogenicity of BI 695501 Versus Humira® in Patients With Moderate to Severe Chronic Plaque Psoriasis	BI 695501 (ADA Biosimilar) vs. ADA (HUMIRA®)	318	PASI75	16	UA	UA	UA	UA	UA
NCT0295153368	Janssen-Cilag (June 2018)	POLARIS	Guselkumab vs. Fumaric Acid Esters	119	PASI90	24	UA	UA	UA	UA	UA
NCT0325538269	AbbVie (Jul 2018)	A Study to Assess the Efficacy of RIS Compared to FUMADERM® in Subjects With Moderate to Severe Plaque Psoriasis Who Are Naïve to and Candidates for Systemic Therapy	RIS vs. Fumaderm	110	PASI90	24	UA	UA	UA	UA	UA
NCT0234624070	UCB Biopharma (Dec 2018)	CIMPACT	Certolizumab Pegol 200mg vs. Certolizumab Pegol 400mg vs. ETA vs. PBO	559	PASI75	12	UA	UA	UA	UA	UA
NCT0266058071	EMD Serono (Aug 2020)	MSB11022 in Moderate to Severe Chronic Plaque Psoriasis	MSB11022 (ADA Biosimilar) vs. ADA (HUMIRA®)	458	PASI75	16	UA	UA	UA	UA	UA
NCT0321943772	AbbVie (Sept 2021)	A Study Comparing the Safety and Efficacy of RIS to Methotrexate in Subjects With Moderate to Severe Plaque Psoriasis	RIS vs. MTX	100	PASI90	28	UA	UA	UA	UA	UA
Table 4C. Unpublished: Study Terminated
NCT0190078273	Novartis (2013)	Efficacy and Safety Study of Subcutaneous SEC in Treatment of Subjects With Moderate to Severe Chronic Plaque-type Psoriasis as Compared to ETA and PBO	SEC vs. ETA vs. PBO	0	PASI	12	UA	UA	UA	UA	UA
NCT0278673274	MedDerm Associates (2017)	Phase 3 Study to Evaluate the Efficacy and Safety of Induction and Maintenance Regimens of Brodalumab Compared With PBO and UST in Subjects With Moderate to Severe Plaque Psoriasis	Brodalumab 210mg vs. Brodalumab 140mg vs. PBO vs. UST	15	PASI	12	UA	UA	UA	UA	UA
NCT0193668875	Merck (2018)	A Study to Evaluate the Efficacy and Safety/Tolerability of Subcutaneous MK-3222 in Participants With Moderate-to-Severe Chronic Plaque Psoriasis (MK-3222–012)	MK-3222 200mg vs. MK-3222 100mg vs. ETA 50mg vs. PBO	0	PASI75	12	UA	UA	UA	UA	UA

The italicized numbers are the observed data, instead of the expected response rates and sample size required for power analyses decided a priori.

Abbreviations: Bi-weekly (BIW); every 2 weeks (Q2W); every 4 weeks (Q4W); every 6 weeks (Q6W); every 8 weeks (Q8W); every 12 weeks (Q12W); Once daily (OD); Adalimumab (ADA); Etanercept (ETA); Infliximab (INF); methotrexate (MTX); Not reported (NR); placebo (PBO); Risankizumab (RIS); Secukinumab (SEC); subcutaneous (SC); unavailable data point (UA); ustekinumab (UST).

^aUsing a modified rating of individual studies from the Oxford Centre for Evidence-based medicine,⁴² 1 indicates a properly powered and conducted randomized clinical trial.

^bStudy was not powered to compare experimental vs. active comparator. Power analyses were reported for the experimental treatment vs. placebo only

Risk of bias

Often, systematic reviews include an assessment of the risk of bias which involves rating each RCT on several domains such as randomization, blinding, and missing data.^14,15 However, a formal risk of bias assessment was not applicable to our study outcome as we focused on specific details in statistical analyses and publication.

Results

We identified 49 psoriasis RCTs for oral and biologic therapies with active comparators. Most were phase 3 trials; however, there was one phase 2 trial¹⁶ and four RCTs^17–20 that did not specify the trial phase. Of the 49 ACTs, there were 13 superiority studies (Table 1), 8 non-inferiority studies (Table 2), 7 equivalence trials (Table 3), and 21 were “other head-to-head” comparisons (Table 4) that could not be classified into one of these 3 categories based on our methodology. The “other head-to-head” studies did not use appropriate wording, or statistical parameters necessary to ascertain the a priori design of the trial. We were also unable to obtain this information after requests to the study sponsors and corresponding authors.

Table 2.

Characteristics of the Non-Inferiority Randomized, Controlled Trials with an Active Comparator.

Source	Sponsor (Completion Year)	Title	Experimental	Active Comparator	PBO	Sample Size (Experiment/Comparator/PBO) A priori/Observed	Primary Outcome	Week Measured	Response Rate A priori/Observed	CI (%)a	Margin (%)	Power	Reported Analyses	Margin Justified	Study Qualityb
Table2A. Non-inferiority Trials with complete power analyses data.
NCT00235820 43,44	Abbott (2008)	CHAMPION	ADA	MTX	PBO	250 (2:2:1) 271 (108/110/53)	PASI75	16	62% ADA vs. 60% MTX	95	−20	0.80	ITT, PP	No	1
NCT0135857845	Novartis (2013)	FIXTUREc	SEC (AIN457) – 150mg & 300mg	ETA	PBO	1264 (1:1:1:1) 1305 (327/323/323/324)	PASI75	12	55% SEC vs. 50% ETA	98.75	−10	0.90	ITT	Yes, prior studies	1
NCT0124159146	Pfizer (2015)	Tofacitinib vs. ETA or PBO	Tofacitinib – 5mg & 10mg	ETA	PBO	1101 (329/330/335/107)	PASI75	12	39.5% 5mg vs. 63.6% 10mg vs. 58.8% ETA vs. 5.6% PBO	95	−15	0.95	ITT	No	1
NCT0172693347	Almirall (2015)	BRIDGE	LAS41008 (DMF)	Fumaderm® (LASW1835)	PBO	690 (2:2:1) 671 (267/273/131)	PASI75	16	50% DMF vs. 35% Fumaderm®	95	−15	0.9	ITT, PP	No	1
NCT0256180648	Eli Lilly and Co. (2017)	IXORA-Sd	Izekizumab	UST	-	300 (1:1) 302 (166/136)	PASI90	12	70% IXE vs. 43% UST	95	−12.6	>0.95	ITT	Yes, “in-house data”	1
Table2B. Non-inferiority Trials with incomplete power analyses data.
NCT0159724549	Eli Lilly and Co. (2019)	UNCOVER-2e	Ixekizumab – Q2W & Q4W	ETA	PBO	1225 (2:2:2:1) 1224 (351/347/358/168)	PASI75	12	89.7% IXE Q2W vs. 77.5% IXE Q4W vs. 41.6% ETA vs. 2.4% PBO	97.5	−12	NR	ITT	No	UA
NCT0164617749	Eli Lilly and Co. (2019)	UNCOVER-3e	Ixekizumab	ETA	PBO	1225 (2:2:2:1) 1346 (385/386/382/193)	PASI75	12	87.3% IXE Q2W vs. 84.2% IXE Q4W vs. 53.4% ETA vs. 7.3% PBO	97.5	−12	NR	ITT	No	UA
Table2C. Unpublished Non-Inferiority Trial
NCT0309010050	Janssen (2018)	ECLIPSE	Guselkumab & matched-PBO	SEC	-	1048	PASI90	48	UA	UA	−10	UA	UA	No	UA

The italicized numbers are the observed data, instead of the expected response rates and sample size required for power analyses decided a priori.

Abbreviations: Bi-daily (BID); Bi-weekly (BIW); every 2 weeks (Q2W); every 4 weeks (Q4W); every 6 weeks (Q6W); Once daily (OD); Adalimumab (ADA); Confidence Interval (CI); Etanercept (ETA); Intention-To-Treat (ITT); Ixekizumab (IXE); Methotrexate (MTX); not reported (NR); Per-Protocol (PP); placebo (PBO); Secukinumab (SEC); subcutaneous (SC); unavailable data point (UA); ustekinumab (UST).

^aAlthough in non-inferiority trials, only one boundary (one-sided CI) of the classical two-sided CI is used for analysis.

^bUsing a modified rating of individual studies from the Oxford Centre for Evidence-based medicine,⁴² 1 indicates a properly powered and conducted randomized clinical trial.

^cA closed-testing procedure used to evaluate different hypotheses. The non-inferiority testing was followed by superiority testing between experimental and placebo.

^dFirstly, evaluated non-inferiority and secondly evaluated superiority of ixekizumab to ustekinumab, as measured by the proportion of patients achieving a PASI 90 response at week 12

^eStudy power was not identified for non-inferiority, only for superiority as >0.93 and therefore, may result in an underpowered study for testing non-inferiority. This study was classified as non-inferiority because the non-inferiority hypothesis was required to be satisfied prior to superiority testing.

Table 3.

Characteristics of the Equivalence Randomized, Controlled Trials with an Active Comparator (none had placebo control).

Source	Title	Author/Sponsor (Completion Year)	Experimental	Active Comparator	Sample Size (Experiment/Comparator/PBO) A priori/Observed	Primary Outcome	Week Measured	Response Rate A Priori/Observed	CI (%)	Margin	Power	Reported Analyses	Margin Justified	Study Qualitya
Table 3A. Equivalence Trials with complete power analyses data.
NTR74320	Fumarates vs. MTX in moderate to severe chronic plaque psoriasis	Fallah Arani et al (2009)	MTX	Fumarate	50 (1:1) 60 (30/30)	Mean change from baseline PASI	12	7.8 decrease (MTX) vs. 7.6 decrease (Fumarate)	95	±5 points in mean PASI	0.90	ITT	No	1
NCT0189186451	EGALITY – Study 302	Sandoz (2015)	GP2015	ETA (ENBREL®)	546 (1:1) 531 (264/267)	PASI75	12	73.4% (GP2015) vs. 75.7% (ETA)	95	±18%	0.90	ITT, PP	Yes, prior studies	1
NCT0197048852	Study to Compare Efficacy and Safety of ABP 501 and ADA (HUMIRA®)	Amgen (2016)	ADA-atto (Amjevita)	ADA (HUMIRA®)	340 (1:1) 350 (175/175)	% Improvement in PASI from Baseline	16	80.9% (ABP501) vs. 83.1% (ADA)	95	±15%	0.90	ITT	No	1
NCT0276295553	CALYPSO	Biocad (2016)	BCD-057	ADA (HUMIRA®)	344 (1:1)	PASI75	16	71% (BCD-057) vs. 71% (ADA)	95	±14%	0.80	NR	NR	1
Table 3B. Equivalence Trials with incomplete power analyses data.
NCT0213421054,55	Comparison of CHS-0214 to Enbrel (ETA) in Patients With Chronic Plaque Psoriasis (PsO)	Coherus Biosciences (2016)	CHS-0214	ETA (ENBREL®)	521 (1:1) 456 (228/228)	PASI75b	12	64.5% (CHS-0214) vs. 62.3% (ETA)	95	±18%	NR	NR	NR	UA
						Mean % change in PASIb	12	76.7% (CHS-0214) vs. 73.4% (ETA)	95	±12.5%	NR	NR	NR	UA
NCT0201610556	ADACCESS	Sandoz (2016)	GP2017	ADA (HUMIRA®)	421 (231/190)	PASI75b	16	67.0% (GP2017) vs 62.4% (ADA)	90	±18%	NR	NR	NR	UA
Table3C. Unpublished Equivalence Trial
NCT0271432257	MYL-1401A Efficacy and Safety Comparability Study to Humira®	Mylan (2017)	MYL-1401A	ADA (HUMIRA®)	294 (2:1)	PASI % change from Baselinec	12	UA	UA	UA	UA	UA	UA	UA

The italicized numbers are the observed data, instead of the expected response rates and sample size required for power analyses decided a priori.

Abbreviations: Bi-weekly (BIW); every 2 weeks (Q2W); every 4 weeks (Q4W); every 6 weeks (Q6W); Once daily (OD); Adalimumab (ADA); Confidence Interval (CI); Etanercept (ETA); methotrexate (MTX); not reported (NR); placebo (PBO); subcutaneous (SC); unavailable data point (UA).

^aUsing a modified rating of individual studies from the Oxford Centre for Evidence-based medicine,⁴² 1 indicates a properly powered and conducted randomized clinical trial.

^bThe secondary endpoint was percent change in PASI from baseline at week 16, with 90% CI and ±15% margins.

^cPrimary outcome previously PASI75 at Week 16

Of particular interest is the ambiguous reporting of margins or delta, as this directly impacts the interpretation of results. Of the 49 studies, 1 of 13 (0.08%), 8 of 8 (100%), and 6 of 7 (85.17%) of the superiority, NI, and equivalence trials, respectively, reported the margin explicitly (i.e., the reader did not have to perform power analyses or calculations to derive missing information). However, in 5 of the superiority trials, we were able to manually calculate the delta (otherwise known as the effect size or acceptable treatment difference).

Our findings indicate that the acceptable difference between treatment arms range from 14% to 20% for superiority trials for PASI75, PASI90, or PASI100 endpoints. The NI studies reported lower bound margins ranging from −20% to −10% for PASI75 or PASI90 endpoints. The equivalence trials reported upper and lower margins ranging from ±12.5% to ±18% for PASI75 or PASI change (mean or percent) from baseline endpoints. Of which, one equivalence RCT reported ±5 points in the mean PASI score. None of the “other head-to-head” trials reported margins or acceptable differences between treatment arms.

Discussion

This systematic review demonstrated considerable heterogeneity in the conduct of head-to-head dermatology trials, reflective of underlying variations in protocol parameters. Although several regulatory bodies have provided lengthy guidance documents for head-to-head trial design^21–23 and reporting,^24,25 our results revealed discrepancies in the conduct and reporting of ACTs in dermatology. Regarding study conduct, there was a diverse range of acceptable treatment differences and inconsistency in the choice of statistical analyses. Inconsistency in the choice of statistical analyses is not problematic per se, as this should be dictated by the primary study question (i.e., establishing non-inferiority, equivalence, or superiority). More problematic, however, is variation in clinical trials reporting. For example, there was a high proportion of studies that omitted data, had ambiguity in power analyses, and violated guidelines for good reporting.²⁴ Indeed, only one of the 8 NI trials successfully indicated non-inferiority study design in the title of the manuscript – the first component of the CONSORT guidelines. Our findings indicate a lack of standardization in study design and reporting, which is critical to allow for appropriate comparison and interpretation of results. Clinical Trials study design and reporting are fundamental because even the slightest change in any study parameters could result in entirely different study conclusions and determination of treatment efficacy.

Studies used language implying either superiority (such as “superior to”), non-inferiority or equivalence RCT design but had much variation in the margins and choice of analyses. Designing ACTs requires a delicate balance of utilizing parameters from current literature and expertise from clinical and research specialists to determine clinically acceptable measures, otherwise known as the margin of error or delta in clinical trials. Our review found a wide range of acceptable treatment differences in superiority, non-inferiority, and equivalence trials (range: 14 to 20%, −20% to −10%, and ±12.5% to ±18%, respectively). Interpretation and application of the margin or delta influences clinical decision making as healthcare providers and patients may differ in their opinions of the maximum acceptable difference between interventions. For example, if the true PASI90 response rates were 60% for an active control and 50% for an investigative treatment, this would be considered as sufficiently similar responses in an equivalence study with ±10% margins. Although one healthcare professional may consider this to be sufficiently “equal” and comfortably prescribe the investigative drug as a replacement, another physician might require more conservative margins.

Readers should also be made aware that the choice of statistical analyses can bias study conclusions. For superiority trials, the intention-to-treat analyses imply a conservative effect on the outcome of the trial. However, for non-inferiority and equivalence trials, ITT analyses do not have the same conservative effect, and it is unclear if per protocol analyses have a conservative effect. Therefore, it is recommended to conduct both intention-to-treat (full analysis set) and “as treated” (per protocol) analyses to support research findings.²⁶ Without both analyses supporting study conclusions in NI and equivalence trials, readers should be cautious of the comparative efficacy conclusions. Only 28.57% (2/7) of the completed NI dermatology trials performed both intention-to-treat and per protocol analyses, which is comparatively worse than another systematic review inclusive of other fields of medicine which found 44% of NI trials conducting both analyses.²⁷ Of the three full-text manuscripts for equivalence trials, only 1 performed both ITT and per protocol analyses. This demonstrates a lack of handling of missing data using more modern and statistically valid approaches.

In addition to the above, details of power analyses and margin rationale were omitted and inconsistent,²⁵ further complicating interpretation of study findings. The International Committee of Medical Journal Editors states that statistical methods should be described with “enough detail to enable a knowledgeable leader with access to the original data to verify the reported results.”²⁸ However, only 7 of the 13 superiority trials reported all of the necessary information to calculate the hypothesized difference between treatments. Of these, only one trial explicitly stated the difference of treatment effect²⁹ whereas we, the authors, used the reported statistical parameters to find this “acceptable difference” for the other trials. In our review, the non-inferiority margin was explicitly stated in all of the trials (8/8, 100%) which is similar to previous findings from Rehal et al.³⁰ reporting 98% (164/168) margin specification. 6 of 7 equivalent studies (85.71%) reported the margins, which is comparable to another study quoting 36 of 42 (86.00%).²⁷ Critically, there was a lack of transparency as we could not make definitive conclusions about 21 trials (classified as “other head-to-head comparisons”) due to missing data. Although a high proportion of these is ongoing or unpublished, the lack of transparency should not be minimized as these parameters are typically decided a priori.

Margin rationale can involve utilizing historical data and clinical judgment as recommended by the International Conference on Harmonisation.²³ However, our findings for psoriasis ACT margin justification are comparatively worse than other fields of medicine. For example, a recent systematic review of NI trials indicated that less than half of the publications reported clinical or statistical justification for the margin of error.³⁰ In our review, 2 of the 8 (25.0%) NI trials and only one of the equivalence trials (1/7, 14.29%) presented reason for margin selection as opposed to 48% (80/162) and 50% (21/42), respectively.²⁷ Our findings are similar to other studies that reviewed margin specification but in other aspects lacked compliance with study conduct and methodology reporting.

As with all studies, our systematic review has limitations. Our results should be approached with caution as there were relatively fewer studies to derive conclusions from (as compared to other fields of medicine). Furthermore, restricting medications to only oral and biologic therapies may have limited our study findings. Lastly, we did not conduct a formal meta-estimate. Typically, meta-analyses utilize observed response rates to estimate the effect of treatment size. However, in this case, it would be inappropriate to conduct a formal meta-analysis based on arbitrarily determined numbers (by clinical trialists) and pooling such findings may not lead to more certain outcomes.³¹ Finally, we were unable to obtain information as described above despite repeated efforts to contact sponsors.

Our findings show incongruity in ACT conduct and reporting which is required for proper evaluation of research findings. Clinicians should be made aware of the different types of study design and of varying acceptable treatment differences, from which conclusions are derived. Readers should be wary of studies that do not report necessary data to reproduce information that is required to determine acceptable differences. Though treatment differences utilized in clinical trials depends upon clinical expertise, there is currently no clear consensus of acceptable treatment differences for ACTs in psoriasis. The lack of agreement makes it difficult to interpret and apply ACT results. We recommend publishing an international consensus statement for various dermatological comparative study designs that include multiple-arm clinical trials (i.e., experimental, active control, and placebo) and suggestions of clinically acceptable differences between treatment options. In the meantime, we strongly urge authors to explicitly state study parameters and communicate consistently per the CONSORT statement for RCT reporting. Lastly, we suggest that sponsors may want to participate by increasing transparency and openly reporting study parameters that are decided a priori.

Conclusion

Considering the impact of these trials on evidence-based medicine, it is imperative for clinicians to understand the variety of statistical measures to interpret research findings appropriately. Treatment differences that appear to be acceptable to clinical trials range from 14% to 20% for superiority, −20% to −12% for non-inferiority and ±12.5% to ±18% for equivalence active comparator trials. The results demonstrate the need for standardized ACT design and clarity in reporting clinical research findings. A better understanding of identifying acceptable margins in the field of dermatology may help future comparison study design.

Figure 1c.

Equivalence Clinical Trial Principles illustrated by different scenarios (1–5), where the effects of an active control and experimental treatment are within the confidence intervals (along the x-axis).

1) Experimental is equivalent to active control because it is within both the upper and lower margins.

2) Experimental is equivalent to active control because it is within both the upper and lower margins.

3) Inconclusive because the CI has values inside and outside the equivalence range.

4) Experimental is not equivalent to active control. It is superior to active control.

5) Experimental is not equivalent to the new treatment and is worse than the active control.

Figure 2.

Evidence Search and Selection.