Figure - PMC

Skip to main content

An official website of the United States government

Here's how you know

Here's how you know

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

View full-text article in PMC

. 2025 Oct 15;16:1592658. doi: 10.3389/fpsyg.2025.1592658

Search in PMC
Search in PubMed
View in NLM Catalog
Add to search

Copyright © 2025 Karvelis and Diaconescu.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

PMC Copyright notice

Graph A shows the relationship between normalized observed effect size and reliability (ICC) for true correlation values of 0.5 and 0.9. Three effect size measures are compared: Pearson's r, Cohen’s d, and Rank-biserial. Graph B depicts the effect of reliability on p-values for a true correlation of 0.5 with a sample size of 60, showing a decrease in p-value as reliability increases. Graph C displays the required sample size across varying reliability for a true correlation of 0.5, alpha of 0.05, and power of 0.8, with sample size decreasing as reliability improves. — Test-retest reliability effects across different effect size metrics and statistical tests. (A) The observed effect sizes as a function of reliability for r_true = 0.5, comparing group differences to correlational strength. Note, because the effect sizes among the tests are not directly comparable, each effect size is normalized by its own maximum value at ICC = 1. The inset shows the results for r_true = 0.9. The dashed line denotes $r_{o b s e r v e d} = r_{t r u e} \sqrt{I C C_{x} I C C_{y}}$ . (B) The p-value as a function of reliability for r_true = 0.5 and the total sample size of N = 60. Dichotomizing data substantially increases p-values, especially when reliability is low. (C) The required sample size to achieve 80% statistical power at α = 0.05 as a function of reliability for the three effect size metrics. Dichotomizing data substantially increases the required sample sizes to detect the same true effect, especially when reliability is low.