Table 5. Caliper tests for exactly recalculated and exactly reported p-values per journal, including alternative binwidths.
Binwidth | 0.00125 | 0.0025 | 0.005 | 0.01 | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
x | N | Pr | p | x | N | Pr | p | x | N | Pr | p | x | N | Pr | p | |
All | 456 | 809 | 0.564 | <.001 | 809 | 1,617 | 0.5 | 0.5 | 1,617 | 3,403 | 0.475 | 0.998 | 3,403 | 7,402 | 0.46 | 1 |
DP | 46 | 87 | 0.529 | 0.334 | 87 | 185 | 0.47 | 0.811 | 185 | 358 | 0.517 | 0.281 | 358 | 756 | 0.474 | 0.932 |
FP | 15 | 27 | 0.556 | 0.351 | 27 | 87 | 0.31 | >.999 | 87 | 192 | 0.453 | 0.915 | 192 | 437 | 0.439 | 0.995 |
JAP | 8 | 20 | 0.4 | 0.868 | 20 | 29 | 0.69 | 0.031 | 29 | 65 | 0.446 | 0.839 | 65 | 141 | 0.461 | 0.844 |
JCCP | 43 | 78 | 0.551 | 0.214 | 78 | 161 | 0.484 | 0.682 | 161 | 364 | 0.442 | 0.988 | 364 | 780 | 0.467 | 0.971 |
JEPG | 27 | 50 | 0.54 | 0.336 | 50 | 98 | 0.51 | 0.46 | 98 | 209 | 0.469 | 0.834 | 209 | 479 | 0.436 | 0.998 |
JPSP | 184 | 305 | 0.603 | <.001 | 305 | 547 | 0.558 | 0.004 | 547 | 1,117 | 0.49 | 0.764 | 1,117 | 2,451 | 0.456 | >.999 |
PLOS | 76 | 149 | 0.51 | 0.435 | 149 | 323 | 0.461 | 0.926 | 323 | 698 | 0.463 | 0.978 | 698 | 1,470 | 0.475 | 0.975 |
PS | 57 | 93 | 0.613 | 0.019 | 93 | 187 | 0.497 | 0.558 | 187 | 400 | 0.468 | 0.912 | 400 | 888 | 0.45 | 0.999 |
Notes.
- x
- frequency of p-values in .05 minus binwidth through .05
- N
- total frequency of p-values across both intervals in the comparison, Pr, x∕N
- p
- p-value of the binomial test
Significant results (α = .05, one-tailed) indicating excess of p-values just below .05 and are reported in bold.