. 2021 Jun 8;4(1):29. doi: 10.5334/joc.161

Table 7.

Summary of Experimental Design from Research Questions to Results.


QUESTION	HYPOTHESIS	TEST NO	ANALYSIS PLAN	POWER ANALYSIS	RESULTS

Is there a time-invariant illusory truth effect?	H1: We will observe a main effect of repetition averaging across all four delay durations.	1	Fit a cumulative link mixed model (as detailed in the “Simulated Data & Analyses” component on the OSF) and conduct χ² test with one degree of freedom, with α = .05.	95% power to detect an effect of .07 or larger on the log odds scale (about a twentieth of a scale point on a seven-point scale). Based on 440 participants completing phase 4.	Supporting H1, there was a significant main effect of repetition when collapsing over interval, $\hat{β_{R}} = 0.57$ (SE = 0.04), χ²(1) = 171.88, p < .001.

		2	IF tests 1 and 3 are non-significant: Test for the absence of the main effect using an equivalence test with bounds of Δ_L = –0.14 and Δ_U of 0.14 on a log odds scale.	95% power to reject the null of a raw effect greater than .085.

Does the illusory truth effect vary over time?	H2: We will observe a repetition-by-interval interaction such that the size of the illusory truth effect will differ across the delay durations.	3	Fit a cumulative link mixed model (as detailed in the “Simulated Data & Analyses” component on the OSF) and test the repetition-by-interval interaction using a χ² test with three degrees of freedom and α = .05.	95% power to detect an effect of a tenth of a scale point, (about.14 on the log odds scale) between two arbitrarily chosen time points: If an illusory truth effect only emerges at very the last time point, we can detect it with 95% power as long as it is at least a tenth of a scale point. Based on 440 participants completing phase 4.	Supporting H2, there was a significant repetition-by-interval interaction, ${\hat{β}}_{R : I 1} = - 0.47$ (SE = 0.05; immediately vs. one day), ${\hat{β}}_{R : I 2} = - 0.67$ (SE = 0.07; immediately vs. one week), ${\hat{β}}_{R : I 3} = - 0.84$ (SE = 0.07; immediately vs. one month), χ²(3) = 121.15, p < .001.

		4	IF test 3 is significant: Use `emmeans()` to attempt to localise the effect, testing the effect at each of the four intervals, and using a Holm-Bonferroni stepwise procedure to keep the familywise error rate at .05.	N/A	Pairwise comparisons revealed that at every interval, estimated marginal means for repeated statements were significantly higher than those for new statements, indicating that the illusory truth effect was present at all four phases (Table 6).

		5	IF test 3 is non-significant: Test for the absence of an interaction effect using an equivalence test considering all six possible pairwise comparisons of the illusory truth effect across intervals to see whether they fall within the bounds of Δ_L = –0.14 and Δ_U of 0.14 on a log odds scale	With \|Δ\| =.14, 37% power to reject H₀ if the true value is 0, about 18% power if true value is .07 or smaller. With \|Δ\| =.20, 93% power if the true value is 0, 75% power if the true value is .07 or smaller, 18% power if the true value is .14 or smaller. For results with .14 < \|Δ\| < .20, see equivtest.html in the repository.