Table 1.
Concept | |
Questionable research practices | Questionable research practices are defined as ‘Design, analytical or reporting practices that have been questioned because of the potential for the practice to be employed with the purpose of presenting biased evidence in favour of an assertion”.70 |
Selective reporting/ cherry picking | The decision about whether to publish a study or parts of a study is based on the direction or statistical significance of the results.71 72 Pre-registration and Registered Reports may prevent selective reporting,26 73 which is also known as cherry picking. |
Publication bias | The decision about whether to publish research findings depends on the strength and direction of the findings.74 The odds of publication are nearly four times higher among clinical trials with positive findings, compared with trials with negative or null findings.75 |
Outcome reporting bias | Only particular outcome variables are included in publications and decisions about which variables to include are based on the statistical significance or direction of the results.71 Outcomes that are statistically significant have higher odds of being fully reported than non-significant outcomes.76 77 |
Attrition bias | Attrition refers to reductions in the number of participants throughout the study due to withdrawals, dropouts or protocol deviations. Attrition bias occurs when there are systematic differences between people who leave the study and those who continue.78 For example, a trial shows no differences between two treatments. In one group, however, half of the participants dropped out because they underwent surgery due to worsening symptoms. |
Null hypothesis statistical testing (NHST) | NHST is originally based on theories of Fisher and Neyman-Pearson. The null hypothesis is rejected or accepted depending on the position of an observed value in a test distribution. While NHST is standard practice in many fields, the International Committee of Medical Journal Editors warns against the inappropriate use and sole reliance on NHST due to several shortcomings of using this approach inappropriately.79 |
p-Hacking | Describes the process of analysing the data in multiple ways until statistically significant results are found. |
HARKing | HARKing, or hypothesising after results are known, is defined as presenting a post-hoc hypothesis as if it were an a priori hypothesis.80 |