Skip to main content
. Author manuscript; available in PMC: 2024 Nov 22.
Published before final editing as: Psychol Methods. 2024 Nov 14:10.1037/met0000695. doi: 10.1037/met0000695

Table 4. Summary of Common Pitfalls Identified in the Literature Review.

Step Pitfalls
Aims Not reporting specific aims
Data-generating mechanism Not summarizing simulation conditions and data-generating mechanism in a structured way (e.g., bullet points, tables)
Not providing justification and Monte Carlo uncertainty coupled with a small number of simulation repetitions
Estimands and other targets Not defining estimands / targets clearly, especially in models with many parameters
Methods Not clearly listing all of the compared methods and their specifications
Performance measures Not clearly defining performance measures
Not clearly defining how performance measures are aggregated
Not reporting Monte Carlo uncertainty
Not reporting convergence
Computational aspects Not reporting computational environment (operating system, software, and package versions)
Not using persistent repositories for sharing code and data (e.g., publisher or university repositories)
Not sharing code and data

Note. Pitfalls were not all coded explicitly, but summarized from the quantitative results of the literature review and discussions between the reviewing authors.