Develop data standards |
Develop data and meta-data |
Produce new datasets |
Capture journal club and conference appraisal data |
|
Leverage electronic health record for data-driven appraisal |
|
Automate appraisal |
Aggregate existing datasets |
Published literature (eg, journal comments, reviews, meta-research, article introductions) |
|
Unindexed sources (eg, PPPR platforms, social media) |
Develop appraisal ontology |
Domain expert–driven |
|
Annotation-derived development |
|
Text-based concept detection |
Expand tools and platforms |
Researcher and practitioner engagement |
|
Patient and research participant engagement |
|
Retrieval tool development |
|
Recommendation system development |
|
Visualization tool development |
Research appraisal knowledge |
Methodology development |
|
Quality assessment |
|
Outcomes research |
Enable other translational research |
Study design, planning, and prioritization |
|
Trial methods development |
|
Novel application to nontraditional evidence sources |
|
Meta-research |
|
Systematic review development |
|
Practice/policy guideline development |
Provide leadership and incentives |
Working group formation |
|
Career, social, and financial incentivization for appraisal production |
|
Cultural norms for appraisal production and uptake |
|
Research funding |
|
Infrastructure development |