Skip to main content
. Author manuscript; available in PMC: 2018 Nov 1.
Published in final edited form as: Psychol Sci. 2017 Sep 12;28(11):1531–1546. doi: 10.1177/0956797617714579

Table 3.

Overview of Sensitivity Analyses for the Effects of Misinformation, Debunking, and Misinformation-Persistence

Effect Sizes All Without Outliers Any Indication of Bias
A. Contoured-Enhanced Funnel Plot: Funnel plots are scatter plots of the effects estimated from individual records against a measure of study size. Asymmetrical funnel plots suggest publication bias (Sterne & Harbord, 2004). Contour lines, which indicate levels of statistical significance (e.g., < .10, < .05,< .01), are added to funnel plots (Peters et at. 2008). FEM was used.
Misinformation Asymmetric funnel plot with records fall outside the funnel Asymmetric funnel plot with records fall outside the funnel Yes, see Figure 2
Debunking Asymmetric funnel plot with records fall outside the funnel - Yes, see Figure 2
Misinformation-Persistence Asymmetric funnel plot with records fall outside the funnel Asymmetric funnel plot with records fall outside the funnel Yes, see Figure 2

B. Trim and Fill Method: A nonparametric method to correct funnel plot asymmetry by removing the smaller records that cause the asymmetry, re-estimating the center of the effect sizes, and filling the omitted records to ensure that the funnel plot is more symmetrical (Borenstein et al., 2009; Duval, 2005). FEM was used.
Misinformation Six estimated records filled on the left Five estimated records filled on the left Yes, see Figure 2
Debunking Zero estimated records filled on the left - No, see Figure 2
Misinformation-Persistence Nineteen estimated records filled on the left Nineteen estimated records filled on the left Yes, see Figure 2

C. Selection Models: The weight-function models accounting for that effect sizes may not have the same probability of being published. Based on different probabilities, a selection model adjusts estimates of the mean effect size and can be compared with the unadjusted one to assess publication bias (Vevea & Woods, 2005). REM was used.
Misinformation Small differences between unadjusted and adjusted estimates Small differences between unadjusted and adjusted estimates No, see Supplementary Information
Debunking Large differences between unadjusted and adjusted estimates - Yes, see Supplementary Information
Misinformation-Persistence Small differences between unadjusted and adjusted estimates Small differences between unadjusted and adjusted estimates No, see Supplementary Information

D. Meta-Regression Model: Publication type can be examined formally as in a moderator analysis. When data are selectively reported in a way that is related to the magnitude of the effect size (e.g., when results are only reported when they are statistically significant), such a variable can have biasing effects (Borenstein et al., 2009). REM was used.

Misinformation Publication type was a significant moderator Publication type was a significant moderator Yes, see Supplementary Information
Debunking Publication type was a significant moderator - Yes, see Supplementary Information
Misinformation-Persistence Publication type was a significant moderator Publication type was a significant moderator Yes, see Supplementary Information

E. p-Curve: This method involves plotting the distribution of p-values reported in a set of studies. The analysis combines the half (.25) and full p-curve to make inferences about evidential value (Simonsohn, Simmons, & Nelson, 2015).
Misinformation P-curve was right-skewed P-curve was right-skewed No, see Supplementary Information
Debunking P-curve was right-skewed - No, see Supplementary Information
Misinformation-Persistence P-curve was right-skewed P-curve was right-skewed No, see Supplementary Information

F. p-Uniform: This analysis holds the same underlying assumption that the distribution of the p value under the null hypothesis that the effect size is equal to the true effect size is uniform (van Assen et al., 2015). P method was used.
Misinformation L.pb test was nonsignificant L.pb test was nonsignificant No, see Supplementary Information
Debunking L.pb test was nonsignificant - No, see Supplementary Information
Misinformation-Persistence L.pb test was nonsignificant L.pb test was nonsignificant No, see Supplementary Information