Table 2. Example categorization of sample-level statements.
Category | Non-Significant | No Effect | Ambiguous, Similar, or Small Effect Size |
---|---|---|---|
Description | Reports that there was no significant difference between two conditions, or words to that effect. | A statement that there was not a difference within the sample, when in fact there was—it was just not significant in their analysis. | A statement about the results that neither suggests they were the same, nor that there was no significant difference. |
Examples | There was no significant/detectable difference between X and Y. We did not detect a difference between X and Y (or any other statement implying failing to find a signal within noise). We did not find a significant effect. X was not significantly related to Y. X did not perform significantly above chance. X performed significantly above chance, but Y did not. There were no significant differences between X and Y’s performance. |
There was no difference between X and Y. There was no effect. There was no evidence of an effect. There was no relationship between X and Y. We did not find/observe/see a difference between X and Y. We did not find an effect. We found no evidence of an effect. X performed at chance levels. |
X and Y were similar. There was no large/clear difference between X and Y. There was no large effect of X on Y. |