Skip to main content
. 2025 Oct 15;16:1592658. doi: 10.3389/fpsyg.2025.1592658

Figure 1.

Panel A displays a scatter plot showing test-retest reliability with measurements at T1 and T2. Panel B features a box plot depicting one-sample effects for Condition 1. Panel C contains a box plot with paired-sample effects for Conditions 1 and 2. Panels D, E, and F show heatmaps for test-retest reliability (ICC), one-sample Cohen’s d, and paired-sample Cohen’s d, respectively. Color gradients in heatmaps represent values, with axes labeled as error variance and between-subject variance.

The reliability paradox. Top panels (A–C) illustrate the statistical tests under consideration: (A) test-retest correlation (same measure obtained twice), (B) one-sample test (mean of a single condition compared to zero), and (C) paired-sample test (mean difference between two conditions). Bottom panels (D–F) show how the observed outcomes of these tests depend on the relative contributions of error variance and between-subject variance. Test-retest reliability (D) increases when error variance is minimized and between-subject variance is maximized, whereas observed one-sample and paired-sample effect sizes (E, F) increase when both error and between-subject variances are minimized.