Table 2. Summary of group discussion.
Group | Question | Challenges 1 |
---|---|---|
Subgroup 1: improving
education & training in rigor, reproducibility, and transparency (RRT) |
1. Can RRT-focused statistics and mathematical modeling courses
improve statistical practice? |
1. It would be difficult to isolate and to evaluate the effects of changes to
existing curricula. |
2. Proximal measures related to technical skills might not translate into
improved research practices. | ||
2. Can specialized training in scientific writing improve transparency? | 1. Writing is an abstract science, which would make measuring outcomes
challenging. |
|
2. There are currently limited existing graduate level curricula that pertain
exclusively to writing. | ||
3. Does modality affect the efficacy of RRT-related education? | 1. Feasibility concerns including, cost, time, and other additional
resources needed to facilitate an intervention. |
|
2. Examining heterogeneity requires large and diverse populations, and is
practically difficult. | ||
Subgroup 2: reducing
statistical errors and increasing analytic transparency |
4. Can automation help identify errors more efficiently? | 1. Automation may be technically possible for only certain types of errors. |
2. New programs intended to automate error correction require a certain
level of computer programming expertise. | ||
5. What is the prevalence and impact of errors? | 1. It would be difficult to generalize the prevalence of errors, because
many common errors have field-specific names. |
|
2. Assessing the impact of errors is largely subjective, unless strict
guidelines are agreed upon and adopted. | ||
6. Do error prevention workflows reduce errors? | 1. It would be difficult to determine if workflows are entirely responsible
for reduced error and improved research practice. |
|
2. It may be challenging to identify generalizable workflows that logically
function across disciplines. | ||
7. How do we encourage post-publication error correction? | 1. It would be difficult to implement standard post-publication error
correction guidelines that function effectively across disciplines. |
|
2. There is a hesitancy to embrace error correction as a normal
component of the editorial process. | ||
Subgroup 3: looking outward:
increasing truthfulness and accuracy of research communications |
8. How does 'spin' in research communication affect stakeholders'
understanding and use of research evidence? |
1. The effects of spin in controlled research settings might not generalize
to real-world decisions. |
2. Reviewing and categorizing text is both subjective and time consuming. | ||
9. Do tools to aid writing research reports increase the
comprehensiveness and clarity of research reports? |
1. Although tools could be developed for testing, implementation
challenges could mitigate their effectiveness in practice. |
|
2. Previous guidelines have had minimal impact on reporting quality. | ||
10. Is it possible to inculcate scientific values and norms related to
truthful, rigorous, accurate, and comprehensive scientific reporting? |
1. There are few model interventions to form self-identity. | |
2. There may be limited opportunities and enthusiasm to integrate
values-based education in classes that focus on technical skills. |
1We present here only two of the most salient challenges.