Design analysis indicates Potential overestimation of treatment effects in randomized controlled trials supporting Food and Drug Administration cancer drug approvals

Abstract

Objective:

Statistical significance drives interpretation of randomized controlled trials (RCTs). We examined the type S error risk—claiming a new drug is falsely beneficial—and exaggeration ratio—how estimated effects differ from true effects—to re-emphasize direction and magnitude of treatment effects.

Study Design and Setting:

We systematically reviewed RCTs supporting Food and Drug Administration (FDA) approval of cancer drugs between 2007 and 2016. We extracted data for overall survival (OS), progression-free survival (PFS), and response outcomes from FDA reviews. We estimated type S error risks and exaggeration ratios by considering replicated RCTs of equal size and a range of true effects.

Results:

We analyzed 42 RCTs for 39 approved drugs. Across 38 RCTs reporting OS, the median type S error risk was 0.00% (Q1–Q3, 0.00–0.01%) and 3.56% (0.40–6.74%), for true hazard ratios of 0.7 and 0.9, respectively, indicating confidence in effect direction. The corresponding exaggeration ratios were 1.09 (1.01–1.11) and 1.30 (1.13–1.42), indicating median overestimations of 9% and 30%. Similar results held for PFS and response outcomes.

Conclusions:

The type S error risk and exaggeration ratio provide additional insights into the replicability of RCTs. Our analyses also quantify the winner’s curse, in which pivotal RCTs tend toward overoptimism.

1. Introduction

Approval of new cancer drugs is typically based on the results of pivotal trials [1]. Recent literature has shown that the quality of evidence provided by pivotal trials varies substantially [2–5]. Overall, the design and interpretation of pivotal trials are subject to conventional methods that place emphasis on statistical significance. Pivotal trials are deemed positive when they show a statistically significant difference (P < 0.05) in the primary endpoint between treatment groups. However, a small P-value does not necessarily suggests strong evidence against the null, and similarly, a large P-value does not necessarily favors the null [6,7]. The maintained focus on statistical significance is linked with a number of problematic consequences [8–10]. In particular, concerns have been raised about the reproducibility of trials [6,11,12].

Novel metrics can provide more focus on the direction and magnitude of treatment effect estimates [13–15]. The type S ("sign") error risk is the probability that a statistically significant treatment effect estimate is in the wrong direction as compared to the true effect. The exaggeration ratio is the factor by which the magnitude of the estimated effect differs from the true effect, given that the estimated treatment effect is statistically significant. These new design analysis concepts have been illustrated in low-powered areas of research, such as social and behavioral science, ecological studies, and developmental economics [14,16,17]. Our objective was to explore the utility of these error quantifications in pivotal randomized controlled trials (RCTs) supporting the approval of cancer drugs by the United States Food and Drug Administration (FDA).

2. Methods

We performed a systematic review of pivotal RCTs supporting cancer drugs approved by the FDA between 2007 and 2016. Across included RCTs, we estimated the power, type S error risk, and exaggeration ratio by considering replicated RCTs of equal size across a range of underlying true treatment effects for overall survival (OS), progression-free survival (PFS), and response outcome data.

2.1. Relationship between type S error risk and exaggeration ratio with power and P-value

We described the methods to calculate the type S error risk and exaggeration ratio in Appendix A. We further illustrated the relationships between both type S error risk and the exaggeration ratio with the power and P-value in a modest simulation study (Fig. 1). We show that the type S error risk decreases as power and true treatment effect increase. Across all scenarios, type S error risk is small, except for low-powered trials. Similarly, the exaggeration ratio decreases when power and true treatment effect increase. But, even RCTs with higher power can still show substantial overestimation for true treatment effects. Finally, we show that the type S error risk and exaggeration ratio give additional insight as compared to the P-value. In particular, both RCTs with significant and nonsignificant P-values can yield large exaggeration ratios. In addition, we show that increasing the significance level alpha results in larger type S error risks and smaller exaggeration ratios; however, the changes are minimal. On the other hand, both type S error risk and exaggeration ratio increase substantially as the standard error (SE) on the treatment effect estimate increases. (Appendix Fig. A.1).

Fig. 1.

Comparisons of type S error and exaggeration ratio against power and P value for a range of true treatment effects. Results of 36,000 simulated parallel-group randomized trials: 2,000 RCTs at 6 different sample sizes (n = 50; 100; 200; 300; 400; 500 per group) crossed with three underlying true odds ratios (ORs) of 1.3 (red), 1.5 (green), and 2.0 (blue). The ORs correspond to probabilities of response of p_e = 25% vs. p_c = 20% in the experimental and control groups, 27% vs. 20%, and 33% vs. 20%, respectively. We generated the number of events in each group according to binomial distributions x_e~Bin(n, p_e) and x_c~Bin(n, p_c). The standard error of each simulated study is given by $SE=\sqrt{\frac{1}{x_e}+\frac{1}{n-x_e}+\frac{1}{x_c}+\frac{1}{n-x_c}}$. We used a 0.5 continuity correction for trials with a zero cell count. In graphs on the top, points to the right of the red dashed line yielded a nonstatistically significant result at the 0.05 significance level.

2.2. Trial selection

We identified all FDA-approved cancer drugs between 2007 and 2016 [18–21]. For each drug, we searched for new drug application (NDA) reviews used for the basis of drug approvals that were publicly available via Drugs@FDA (https://www.accessdata.fda.gov/scripts/cder/daf/). We then searched for the statistical review used for each NDA. If not available, we used the medical or summary reviews. We examined the executive summary from each review to determine exclusion of noncancer indications and examined the full-text to identify eligible RCTs.

We included pivotal phase II and III RCTs reporting primary and/or secondary time-to-event or response outcome measures. We excluded supportive RCTs that were not used to determine approval, and any studies that were single-arm, nonrandomized, noncomparative, or RCTs with multiple treatment arms. Noncomparative studies were defined as those in which the control arm treatment was the same drug at a different dosage, or in which the analysis was done separately for each treatment arm.

2.3. Data extraction

The first author extracted data from NDA reviews, and the last author independently compared all extracted data with the reviews. For each RCT, we collected information about the drug indication, trial sample size, randomization ratio, drug name and class, whether the RCT was stopped early, and outcome data. We classified drug indications according to National Cancer Institute classification (https://www.cancer.gov/types/by-body-location).

Time-to-event outcome measures included OS, PFS, event-free survival (EFS), and time to progression (TTP). We extracted the hazard ratio (HR) and associated 95% confidence interval (CI), then converted them into the log HR and associated SE. In two cases, the associated 95% CIs were not reported, so we used the exact P-values to derive the SE. Response outcomes included change in tumor burden (overall/objective response, best overall response, or complete response) or symptom burden (e.g., reduction in spleen volume in myelofibrosis). We extracted the number of events and the number randomized for each treatment arm to calculate the log odds ratio (OR) and associated SE. For trials with zero cell counts, we corrected by adding 0.5 to each cell. For each trial and endpoint, we extracted the P-values, and when not available, we tested for statistical significance by using a two-sided z-test.

In all trials, we used the longest follow-up outcome measures available. In the case of trials that stopped early, we used the results from the interim analysis. When multiple HR estimates were provided for time-to-event outcome measures, priority was given to Cox Proportional Hazards Model estimates over Pike Estimator estimates and adjusted/stratified analyses over unadjusted analyses. If an RCT reported PFS and TTP, we used PFS. If an RCT reported multiple response outcomes, we used the primary outcome measure.

2.4. Estimation of power, type S error risk, and exaggeration ratio

For each RCT, we considered a replicated trial of the same size and calculated the power, type S error risk, and exaggeration ratio for hypothesized true treatment effects (Appendix A). We prespecified a range of underlying true effects shown to be reasonable choices in the context of cancer trials [22–24]. For OS and PFS, we considered the following HR values: 0.70, 0.75, 0.80, 0.85, 0.90, and 0.95. Because response is a positive outcome, we considered the corresponding range of true OR: 1.43, 1.33, 1.25, 1.18, 1.11, and 1.05.

The type S error risk is interpreted as the probability that the replicated trial shows significant benefit with the experimental treatment when the control treatment is truly superior. In addition, an exaggeration ratio > 1 (let us say 1.25) means that the true underlying HR or OR is overestimated (by 25%). In RCTs without evidence of a significant effect, the type S error risk and exaggeration ratio pertain to the results if a replicated RCT of the same size yielded significant findings.

We analyzed OS, PFS, and response outcomes separately. We grouped temporal measures of disease progression together (including PFS, TTP, and EFS) and grouped all response outcomes together. In sensitivity analyses, we estimated the power, type S error risk, and exaggeration ratio in the subgroup of phase II RCTS, in active compound controlled trials, and for primary outcomes only. We performed all analyses using SAS version 9.4 (SAS Institute, Cary, NC) software. An online applet to calculate the type S error risk and exaggeration ratio is available online: https://ludovic.shinyapps.io/designanalysis/.

3. Results

3.1. Trial selection and characteristics

Among 70 available NDA reviews, we included a total of 42 RCTs supporting 40 NDAs for 39 drugs (Fig. 2). Exclusion reasons are reported in Appendix Table B.1. Characteristics of included RCTs are summarized in Table 1 and detailed in Appendix Table B.2. Selected RCTs covered 11 cancer types. The median sample size across RCTs was 465 patients [Q1–Q3, 303–761]. Of the 42 selected RCTs, 5 (12%) were phase II trials, and 2 (5%) were stopped early for efficacy. Of the 42 RCTs, 38 (90%) reported OS; 37 (88%) RCTs reported PFS; and 35 (83%) RCTs reported response outcomes.

Fig. 2.

Flow diagram of trial selection. FDA, Food and Drug Administration; NDA, new drug application.

Table 1.

Characteristics of selected randomized controlled trials

Characteristic	All RCTs, N = 42	OS, N = 38	PFSa, N = 37	Response, N = 35
Primary outcome measure, n (%)		16 (42)	25 (68)	7 (20)
Phase II, n (%)	5 (12)	4 (11)	3 (8)	5 (14)
NDA year, median [Q1–Q3]	2013 [2011–2015]	2013 [2011–2015]	2013 [2011–2015]	2012 [2011–2015]
Cancer type, n (%)
Genitourinary	8 (19)	8 (21)	8 (22)	5 (14)
Breast	6 (14)	5 (13)	6 (16)	6 (17)
Hematologic/Blood	6 (14)	5 (13)	5 (14)	6 (17)
Skin	5 (12)	5 (13)	5 (14)	5 (14)
Lung	4 (10)	4 (11)	3 (8)	2 (6)
Thyroid	3 (7)	3 (8)	3 (8)	3 (9)
Musculoskeletal	3 (7)	2 (5)	1 (3)	3 (9)
Gastrointestinal	2 (5)	2 (5)	2 (5)	1 (3)
Colorectal	2 (5)	2 (5)	2 (5)	2 (6)
Neurologic	2 (5)	1 (3)	1 (3)	1 (3)
Soft tissue	1 (2)	1 (3)	1 (3)	1 (3)
Sample size, median [Q1eQ3]	465 [303–761]	465 [322–768]	495 [322–762]	416 [301–679]
Randomization ratio, n (%)
1:1	22 (53)	20 (53)	19 (51)	19 (54)
2:1	19 (45)	17 (45)	17 (46)	15 (43)
3:1	1 (2)	1 (3)	1 (3)	1 (3)
Experimental arm treatment, n (%)
Single agent	23 (55)	20 (53)	18 (49)	19 (54)
Combination	19 (45)	18 (47)	19 (51)	16 (46)
Control arm treatment, n (%)
Placebo/No treatment	27 (64)	25 (66)	25 (68)	23 (66)
Active compound	15 (36)	13 (34)	12 (32)	12 (34)
Early stop, n (%)	2 (5)	2 (5)	2 (5)	1 (3)

RCT, randomized controlled trial; OS, overall survival; PFS, progression free survival; NDA, new drug application.

^aOne trial reported event-free survival, and two trials reported time to progression.

For the 38 RCTs reporting OS, the median HR was 0.76 [Q1–Q3, 0.68–0.83], and 22 (58%) RCTs showed statistically significant results. For the 37 RCTs reporting PFS, the median HR was 0.61 [0.42–0.70], and 31 (84%) RCTs showed statistically significant results. For the 35 RCTs reporting response outcome measures, the median OR was 3.10 [1.46–12.20], and 25 (71%) showed statistically significant results.

Funnel plots showed that there was no evidence of an association between trial size and estimated effect for OS (P = 0.23; Fig. 3). However, trial size was associated with treatment effect estimates for PFS and response outcomes, with smaller RCTs showing larger effects (P < 0.0001 for both). For all outcomes, active compound controlled trials did not demonstrate systematically different effects as compared to placebo/no treatment controlled trials (metaregression P-value 0.58 for OS; 0.39 for PFS; and 0.86 for response).

Fig. 3.

Funnel plots comparing effect estimates to standard error across outcome measures. Effect estimates and corresponding standard errors were transformed to the log scale. For PFS and response outcomes, trial size was significantly associated with treatment effect estimates, with smaller RCTs showing larger effects (P = 0.00004 and P = 0.00001, respectively). Control arm treatment type (active compound, shown as black diamond vs. placebo/no treatment, shown as white circle) did not demonstrate systematic differences for this relationship across all outcome measures.

3.2. Power, type S error risk, and exaggeration ratio of treatment effects

Power, type S error risk, and the exaggeration ratio of underlying true treatment effects across outcome measures are reported for the 42 trials in Fig. 4 and summarized in Table 2. RCTs with smaller size, that is, larger SEs on the treatment effect estimate, have smaller power, yield larger type S error, and larger exaggeration ratios. Across 38 RCTs reporting OS, for a true HR of 0.70, the median power was 66.38% [Q1–Q3, 47.69–97.77%], the median type S error risk was 0.00% [0.00–0.01%], and the median exaggeration ratio was 1.09 [1.01–1.11]; this means that the chance of concluding that an experimental treatment is beneficial when in fact it is detrimental is improbable, but that a trial result, if statistically significant, overestimates the true HR by 9%, in median. For a true HR of 0.90, these numbers were 10.84% [8.69–21.69%], 3.56% [0.40–6.74%], and 1.30 [1.13–1.42] (i.e., a median overestimation of the true effect on OS by 30%).

Fig. 4.

Scatter plots comparing the estimated power, type S error risk, and exaggeration ratio for a range of true underlying effect sizes across outcome measures. Each point on the plots corresponds to one RCT (OS, N = 38; PFS, N = 37; Response, N = 35). The type S error risk is the probability that the replicated estimate has the incorrect sign, if it is statistically significantly different from the null. The exaggeration ratio is the expectation of the absolute value of the replicated estimate divided by the true effect size, if it is statistically significantly different from zero. An exaggeration ratio >1 means that the true underlying treatment effect is overestimated. For OS and PFS, true underlying effects ranged as follows: 0.70, 0.75, 0.80, 0.85, 0.90, 0.95; for response outcomes, true underlying effects ranged as follows: 1.43, 1.33, 1.25, 1.18, 1.11, 1.05 with lighter shades of blue representing true underlying effects closer to one. OS, overall survival; PFS, progression-free survival.

Table 2.

Power, type S error risk, and the exaggeration ratio across true effect sizes by outcomes

Outcome	True treatment effect	Power (%)	Type S error risk (%)	Exaggeration ratio
Overall survival (N = 38)	HR = 0.70	66.38 [47.69–97.77]	0.00 [0.00–0.01]	1.09 [1.01–1.18]
	HR = 0.80	31.98 [22.16–70.04]	0.09 [0.00–0.38]	1.19 [1.05–1.29]
	HR = 0.90	10.84 [8.69–21.69]	3.56 [0.40–6.74]	1.30 [1.13–1.42]
Progression-free survival (N = 37)	HR = 0.70	79.91 [49.38–94.67]	0.00 [0.00–0.01]	1.05 [1.02–1.17]
	HR = 0.80	41.72 [22.93–60.86]	0.02 [0.00–0.33]	1.13 [1.07–1.28]
	HR = 0.90	13.12 [8.86–18.42]	2.03 [0.70–6.36]	1.23 [1.16–1.40]
Response outcomes (N = 35)	OR = 1.43	17.41 [10.75–47.79]	0.84 [0.01–3.65]	1.71 [1.18–2.49]
	OR = 1.25	9.72 [7.20–22.09]	4.88 [0.37–11.52]	1.89 [1.29–2.77]
	OR = 1.11	6.01 [5.48–8.60]	19.99 [6.90–27.83]	2.09 [1.42–3.05]

Data are median [Q1–Q3]. For example, across the 38 RCTs reporting OS, and for a true underlying effect of 0.70, the median power was 66%, the median type S error risk was 0%, and the median exaggeration ratio was 1.09 (meaning that the true effect is overestimated on average by 9%). Across the 35 RCTs reporting response outcomes, and for a true underlying effect of 1.43, the median power was 17%, the median type S error risk was 0.84%, and the median exaggeration ratio was 1.71 (meaning that the true effect is overestimated on average by 71%).

Across 37 RCTs reporting PFS, the median power was 79.91% [49.38–94.67%], the median type S error risk was 0.00% [0.00–0.01%], and the median exaggeration ratio was 1.06 [1.02–1.17] for a true HR of 0.70. These numbers were 13.12% [8.86–18.42%], 2.03% [0.70–6.36%], and 1.23 [1.16–1.40] for a true HR of 0.90 (i.e., a median overestimation of the true effect on PFS by 23%).

Across 35 RCTs reporting response outcomes, the median power was 17.41% [10.75–47.79%], the median type S error risk was 0.84% [0.01–3.65%], and the median exaggeration ratio was 1.71 [1.18–2.49] for a true OR of 1.43; this means that the trial results, if statistically significant, overestimate the true OR by 71%, in median. These numbers were 6.01% [5.48–8.61%], 19.99% [6.90–27.83%], and 2.09 [1.42–3.05] for a true OR of 1.11. Fig. 4 shows the estimated power, type S error risk, and exaggeration ratio across the range of true underlying treatment effects for each RCT.

3.3. Sensitivity analyses

Power, type S error risk, and the exaggeration ratio of underlying true treatment effects in phase II RCTs are reported in Appendix Table C.1. In four phase II RCTs reporting OS, three phase II RCTs reporting PFS, and five phase II RCTs reporting response outcomes, we found larger type S error and larger exaggeration ratios as compared to the overall results. Among 13 active compound controlled RCTs reporting OS, 12 active compound controlled RCTs reporting PFS, and 12 active compound controlled RCTs reporting response outcomes, results were consistent with the overall results (Appendix Table C.2). Results were also consistent with the overall results in 16 RCTs reporting OS as the primary outcome, 25 RCTs reporting PFS as the primary outcome, and 7 RCTs reporting response outcomes (Appendix Table C.3).

4. Discussion

In our design analysis of 42 pivotal RCTs supporting FDA approval for cancer drugs, we found that, for a range of plausible true treatment effects, trials with larger SEs on the treatment effect were likely to overestimate the true magnitude of effect. For example, for a true HR of 0.80 for OS, one trial of four overestimated the effect by more than 29%. The risk of claiming a new drug is beneficial when it is actually performing worse than the comparator (type S error) was close to zero, except in cases where the true effect was small (e.g., HR on OS of 0.9). Given the observed size of RCTs, the power to detect the plausible true effects considered was frequently degraded. Finally, we found that trial size was significantly associated with effect estimates for PFS and response outcomes. In particular, smaller trials showed larger effects.

Type S error risk and exaggeration ratio results were consistent for PFS and OS. PFS was a primary outcome in 68% of selected RCTs, which reflects the common use of surrogate endpoints in drug trials [2]. Considering that most pivotal trials have follow-up durations of 6 months or less, it further raises the risk of making erroneous and exaggerated inferences. In addition, results for response outcomes suggested even larger exaggeration ratios as compared to PFS and OS. This can be explained in part by the difference in precision of effect estimates. The SE for the log HR, used for time-to-event outcomes, exceeds the SE for the log OR, used for response outcomes.

Our findings offer a quantification of the "winner’s curse" in RCTs [25–28]. A pivotal RCT that "wins" against the statistical significance threshold is likely biased and overoptimistic. RCTs with exaggerated results may have direct detrimental implications because of undue adoption of a drug in clinical practice. Furthermore, such RCTs may affect subsequent RCTs assessing the same drug, in which power calculations may be based on the inflated estimate of the true treatment effect. Thus, subsequent RCTs will be underpowered to detect the true treatment effect, yielding poor reproducibility. Prior works showed that early RCTs may yield inflated treatment effect estimates [25,26,29–31]. Such investigations were possible by conducting a retrospective examination of cumulative evidence over time. The advantage of using the type S error risk and exaggeration ratio is the ability to estimate the reliability of RCT results as they stand on their own.

Our findings also suggest a small-study effect among RCTs supporting drug approval by the FDA. Our funnel plot has limitations because RCTs examined different drugs. Consequently, the asymmetry on the funnel plot may be due to different underlying true effects. However, we did not see differences between RCTs against placebo/no treatment and RCTs against active comparators. This pattern is linked to the design analysis metrics because larger SEs yield increased type S errors and exaggeration ratios.

Other metrics have been proposed to quantify the strength of evidence of RCTs. Van Ravenzwaaij and Ioannidis suggest using Bayes factors (or likelihood ratios) as a routine tool to assess the endorsement of new drugs [32]. The Bayes factor places emphasis on the strength of evidence rather than on statistical significance. However, there is no universally accepted approach to compute Bayes factors, and some methods are misleading [33,34]. Another metric proposed to examine the robustness of RCTs showing statistically significant results is the Fragility Index [35,36]. However, the Fragility Index is strongly associated with the P-value and, consequently, cannot be used as a measure of the strength of the treatment effect [37].

Gelman and Carlin’s design analysis includes a power calculation [14]. In reports of RCTs that fail to show a statistically significant difference, it is still common to find a discussion about the power of the RCT to detect the observed difference. However the so-called post hoc power adds no value to the data analysis [7,38]. If the P-value is larger than 0.05, the post hoc power is smaller than 50%. In contrast, Gelman and Carlin’s power analysis relies on a hypothesized treatment effect informed by a metaanalysis of prior trials, if any, or over a range of plausible values for the true effect [14].

Our analysis has limitations. First, the computation of the type S error risk and exaggeration ratio rely on a hypothesized treatment effect estimate. Gelman and Carlin [14] recommend informing this estimate by external subject-matter knowledge, such as from previous studies. In the context of FDA NDAs, pivotal trials supporting the application are typically the first and only RCTs estimating the treatment effect, with no prior external sources available to reliably inform the hypothesized effect. Another option is to use the expected treatment effect prespecified in the treatment protocol, but hypothesized effects are also commonly overly optimistic [39,40]. We have considered a wide range of treatment effects consistent with effects seen in large cohorts of cancer RCTs unaffected by publication bias [22,24,41]. In fact, Djulbegovic et al. have shown in a comprehensive evaluation of 624 RCTs conducted by the NCI-funded COG program that HR for OS and EFS were on average 0.95 and 0.90, respectively. The median HR for OS among 176 trials with significant findings favoring the experimental treatment was 0.83.

A second limitation is that we quantified the type S error risk and exaggeration ratios for relative measures of effect. The ORs and the HRs were the measures of effect used in the statistical and medical reviews at the FDA. Previous studies show that pivotal trials of approved cancer drugs yielded moderate gain in OS of only about 1–2 months [5,42,43]. As such, it would be helpful to conduct the design analysis for absolute measures of treatment effects. In particular, examination of the exaggeration ratio for the difference in Restricted Mean Survival Times would be useful for cancer RCTs informing the effect of drugs on OS. Finally, we focused on efficacy endpoints, but drug approvals do not rely on efficacy alone. We could derive the type S error risk and exaggeration ratio for safety outcomes and quality of life.

In conclusion, our results suggest that reliance on statistical significance is problematic, as even statistically significant treatment effect estimates are likely overestimated. Design analyses by using the type S error risk and exaggeration ratio offer insights into the design and interpretation of cancer RCT results, particularly in the framework of reproducibility. We recommend that authors report type S error risk and exaggeration ratio over a range of plausible treatment effects.

What is new?

Key findings

• We examined the type S error risk—claiming that a new drug is falsely beneficial—and exaggeration ratio—how estimated effects differ from true effects—to re-emphasize the direction and magnitude of treatment effects.

• Among pivotal randomized controlled trials (RCTs) supporting Food and Drug Administration approval for cancer drugs, RCTs with larger standard errors on the treatment effect estimates were likely to overestimate the magnitude of the true treatment effect.

• We observed this finding consistently across a range of plausible cancer treatment effects for overall survival, progression-free survival, and response outcomes.

What this adds to what was known?

• Design analyses using the type S error risk and exaggeration ratio provide additional insights into the replicability of RCTs, beyond statistical significance.

• Our findings offer a quantification of the "winner’s curse" in RCTs.

What is the implication and what should change now?

• Investigators should report design analyses by using the type S error risk and exaggeration ratio over a range of plausible treatment effects to better inform the interpretation of RCTs.