A. Contoured-Enhanced Funnel Plot: Funnel plots are scatter plots of the effects estimated from individual records against a measure of study size. Asymmetrical funnel plots suggest publication bias (Sterne & Harbord, 2004). Contour lines, which indicate levels of statistical significance (e.g., < .10, < .05,< .01), are added to funnel plots (Peters et at. 2008). FEM was used. |
Misinformation |
Asymmetric funnel plot with records fall outside the funnel |
Asymmetric funnel plot with records fall outside the funnel |
Yes, see Figure 2
|
Debunking |
Asymmetric funnel plot with records fall outside the funnel |
- |
Yes, see Figure 2
|
Misinformation-Persistence |
Asymmetric funnel plot with records fall outside the funnel |
Asymmetric funnel plot with records fall outside the funnel |
Yes, see Figure 2
|
|
B. Trim and Fill Method: A nonparametric method to correct funnel plot asymmetry by removing the smaller records that cause the asymmetry, re-estimating the center of the effect sizes, and filling the omitted records to ensure that the funnel plot is more symmetrical (Borenstein et al., 2009; Duval, 2005). FEM was used. |
Misinformation |
Six estimated records filled on the left |
Five estimated records filled on the left |
Yes, see Figure 2
|
Debunking |
Zero estimated records filled on the left |
- |
No, see Figure 2
|
Misinformation-Persistence |
Nineteen estimated records filled on the left |
Nineteen estimated records filled on the left |
Yes, see Figure 2
|
|
C. Selection Models: The weight-function models accounting for that effect sizes may not have the same probability of being published. Based on different probabilities, a selection model adjusts estimates of the mean effect size and can be compared with the unadjusted one to assess publication bias (Vevea & Woods, 2005). REM was used. |
Misinformation |
Small differences between unadjusted and adjusted estimates |
Small differences between unadjusted and adjusted estimates |
No, see Supplementary Information
|
Debunking |
Large differences between unadjusted and adjusted estimates |
- |
Yes, see Supplementary Information
|
Misinformation-Persistence |
Small differences between unadjusted and adjusted estimates |
Small differences between unadjusted and adjusted estimates |
No, see Supplementary Information
|
|
D. Meta-Regression Model: Publication type can be examined formally as in a moderator analysis. When data are selectively reported in a way that is related to the magnitude of the effect size (e.g., when results are only reported when they are statistically significant), such a variable can have biasing effects (Borenstein et al., 2009). REM was used. |
|
Misinformation |
Publication type was a significant moderator |
Publication type was a significant moderator |
Yes, see Supplementary Information
|
Debunking |
Publication type was a significant moderator |
- |
Yes, see Supplementary Information
|
Misinformation-Persistence |
Publication type was a significant moderator |
Publication type was a significant moderator |
Yes, see Supplementary Information
|
|
E. p-Curve: This method involves plotting the distribution of p-values reported in a set of studies. The analysis combines the half (.25) and full p-curve to make inferences about evidential value (Simonsohn, Simmons, & Nelson, 2015). |
Misinformation |
P-curve was right-skewed |
P-curve was right-skewed |
No, see Supplementary Information
|
Debunking |
P-curve was right-skewed |
- |
No, see Supplementary Information
|
Misinformation-Persistence |
P-curve was right-skewed |
P-curve was right-skewed |
No, see Supplementary Information
|
|
F. p-Uniform: This analysis holds the same underlying assumption that the distribution of the p value under the null hypothesis that the effect size is equal to the true effect size is uniform (van Assen et al., 2015). P method was used. |
Misinformation |
L.pb test was nonsignificant |
L.pb test was nonsignificant |
No, see Supplementary Information
|
Debunking |
L.pb test was nonsignificant |
- |
No, see Supplementary Information
|
Misinformation-Persistence |
L.pb test was nonsignificant |
L.pb test was nonsignificant |
No, see Supplementary Information
|