Fig. 8.

Violin plots of the participant’s perceived confidence in their judgment between sample means (i.e. “which of two candidates will win the election, given the polling data?”), plotted against the actual average p-value of the relevant 2-sample t-test. While, for across all presented graph types, participants’ average confidence was negatively correlated with p-value (R² = 0.66, β = 8.30), unlike in statistical practice (where we would reject as not statistically significant differences with p-values of 0.05 or higher), participants in general become gradually more confident on average with decreases in p-value.