Skip to main content
. 2021 May 19;8(5):201697. doi: 10.1098/rsos.201697

Table 3.

Examples of a true and false positive finding for each metric.

metric false positive true positive
(1) significance an original study and its replication attempt both have a positive observed effect size; the replication attempt has a p-value below 0.05 for a known population effect size of (approximately) 0 an original study and its replication attempt both have a positive observed effect size; the replication attempt has a p-value below 0.05 for a known population effect size greater than 0
(2) Small Telescopes an original observed effect size is positive; the observed replication effect size is not significantly smaller than the small effect size, which is computed based on the observed original effect size for a known population effect size of (approximately) 0 an original observed effect size is positive; the observed replication effect size is not significantly smaller than the small effect size, which is computed based on the observed original effect size for a known population effect size greater than 0
(3) classical meta-analysis an observed original effect size and the fixed-effect meta-analytic effect size are both positive; the meta-analysis has a p-value below 0.05 for a known population effect size of (approximately) 0 an observed original effect size and the fixed-effect meta-analytic effect size are both positive; the meta-analysis has a p-value below 0.05 for a known population effect size greater than 0
(4) BF an original study and its replication attempt both have a positive observed effect size; the BF of the replication study is larger than 1 for a known population effect size of (approximately) 0 an original study and its replication attempt both have a positive observed effect size; the BF of the replication study is larger than 1 for a known population effect size greater than 0
(5) replication BF an original study has a p-value below 0.05; the replication BF is larger than 1 for a known population effect size of (approximately) 0 an original study has a p-value below 0.05; the replication BF is larger than 1 for a known population effect size greater than 0
(6) Bayesian meta-analysis an observed original effect size and the Bayesian meta-analytic effect size are both positive; the meta-analysis has a BF larger than 1 for a known population effect size of (approximately) 0 an observed original effect size and the Bayesian meta-analytic effect size are both positive; the meta-analysis has a BF larger than 1 for a known population effect size greater than 0
(7) sceptical p-value an observed original effect size is positive; the sceptical p-value for an original study and its replication attempt is smaller than 0.025 for a known population effect size of (approximately) 0 an observed original effect size is positive; the sceptical p-value for an original study and its replication attempt is smaller than 0.025 for a known population effect size greater than 0