Table 3.

Pairwise agreement between treatment hierarchies obtained from the different ranking metrics measured by Spearman $\mathit{\rho }$, Kendall $\mathit{\tau }$, Yilmaz ${\mathit{\tau }}_{\mathit{A}\mathit{P}}$ and Average Overlap

	${\mathit{p}}_{\mathit{B}\mathit{V}}$vs${\mathit{S}\mathit{U}\mathit{C}\mathit{R}\mathit{A}}_{\mathit{F}}$	${\mathit{S}\mathit{U}\mathit{C}\mathit{R}\mathit{A}}_{\mathit{F}}$versus relative treatment effect	${\mathit{p}}_{\mathit{B}\mathit{V}}$versus relative treatment effect	${\mathit{S}\mathit{U}\mathit{C}\mathit{R}\mathit{A}}_{\mathit{F}}$versus${\mathit{S}\mathit{U}\mathit{C}\mathit{R}\mathit{A}}_{\mathit{B}}$
Spearman$\mathit{\rho }$	0.9 (0.8 to 0.96)	1 (0.99 to 1)	0.9 (0.8 to 0.97)	1 (0.98 to 1)
Kendall$\mathit{\tau }$	0.8 (0.67 to 0.91)	1 (0.95 to 1)	0.8 (0.69 to 0.91)	1 (0.93 to 1)
Yilmaz${\mathit{\tau }}_{\mathit{A}\mathit{P}}$	0.78 (0.6 to 0.9)	1 (0.93 to 1)	0.79 (0.65 to 0.9)	1 (0.93 to 1)
Average Overlap	0.85 (0.72 to 0.96)	1 (0.91 to 1)	0.88 (0.79 to 1)	1 (0.94 to 1)

Medians, first and third quartiles are reported.

Relative treatment effect stands for the relative treatment effect against fictional treatment of average performance.

P_BV, probability of producing the best value; SUCRA_B, surface under the cumulative ranking curve (calculated in Bayesian setting); SUCRA_F, surface under the cumulative ranking curve (calculated in frequentist setting).