Figure 4.

Illustration of long run False Positive Probability (FRP) and True Positive Probability (TRP) of studies. Let's assume that we run 2 × 100 studies, H₀ is true in 100 studies and H₁ is true in 100 studies with α = 0.05 and Power = 1−β = 0.6. (A) Shows the outcome of true H₀ studies, 5 of the 100 studies coming up statistically significant. (B) Shows the outcome of true H₁ studies, 60 of the 100 studies coming up statistically significant [note that realistically the 60 studies would be scattered around just as in panel (A) but for better visibility they are represented in a block]. (C) Illustrates that true H₀ and true H₁ studies would be indistinguishable. That is, researchers do not know which study tested a true H₀ or true H₁ situation (i.e., they could not distinguish studies represented by black and gray squares). All they know is whether the outcome of a particular study out of the 200 studies run was statistically significant or not. FRP is the ratio of false positive (H₀ is true) statistically significant studies to all statistically significant studies: 5/65 = 0.0769. TRP is the ratio of truly positive (H₁ is true) statistically significant studies to all statistically significant studies: 60/65 = 0.9231 = 1 − FRP = 1 − 0.0769.