Mastering Abstract Review for Medical Education Conferences: Fundamental Tips to Grow and Support Scholars

Abstract

Medical education conferences have become a routine way to disseminate scholarly activity in the form of workshops, posters, or oral reports. Curricular developments, assessment tools, teaching innovations, and faculty development initiatives are just some examples of the submission types to medical education conferences. Just as it is important to uphold rigor and standards to the examination of medical education research publications, it is essential to have guidelines from which medical education conference abstracts are appraised, and there is a lack of written recommendations for this in the literature. We propose 12 tips to guide reviewers of such medical education conference abstracts with our structured approach.

Background

It has become the standard expectation for medical educators to contribute and produce scholarly works for career development and academic promotion [1]. Medical education conferences are one venue that hosts such scholarly works as workshops, symposia, quick shot podium presentations, and posters. The ability to write a great abstract is pivotal to advancement and promotion [2]. As such, scholarly reviewing of conference abstracts is a crucial activity for medical education faculty, serving multiple purposes. What often hinders this process is reviewers’ varying interpretation of what objectivity means for the abstract and to the field as a whole; and thus, it is important to follow a standard [3]. The peer review system overall is often regarded by academic faculty as being nontransparent, arbitrary, and inherently biased [4]. A rigorous peer review process ensures that medical education conferences showcase high-impact, innovative work that drives the field forward, cultivating deeper participation in a scholarly community [5–7]. While the general topic of peer reviewing for journals is prevalent in the medical education literature [8–11], there is a lack of clear guidelines and practical advice for reviewers to overcome barriers and provide actionable feedback to authors [12], as well as address inconsistency in the reports between reviewers [13]. Furthermore, although there have been publications about how to review abstracts for presentations, these focus largely on empirical research reports and not on program descriptions or teaching innovations, which are common submission types at medical education conferences [14, 15].

This 12-tip article aims to close this gap by introducing a structured approach to provide new reviewers with useful advice on how to be successful in this role, with particular emphasis on medical education conferences. By offering practical strategies and insights, we seek to equip novice reviewers with the tools necessary to conduct thorough, fair, and constructive reviews, ultimately enhancing the quality of conference proceedings and contributing to the advancement of medical education scholarship. Furthermore, this guidance will help ensure that the review process is both efficient and effective, fostering a more robust and dynamic academic community.

Tip 1— Ensure the Title Captures the Essence of the Abstract

The title is typically the first thing that participants will see when attending a conference session, so it is essential that it captures its core message. Reviewers can provide important guidance to the author such as avoiding abbreviations and acronyms that participants may not recognize (e.g., specialty-specific terms that are not widely known to the general public). Further, vague titles or the inclusion of unnecessary words or those unrelated to the session should also be addressed. These practices could deter participants from attending a session. Much like writing learning objectives in medical education using frameworks like Bloom’s taxonomy [16, 17], including an action verb in the title to emphasize what the session is about provides potential participants with information on what to expect from the session (Fig. 1).

Fig. 1.

Derived from Anderson and Krathwol’s (2001) revision to Bloom’s cognitive hierarchy

For example, stating “Enhancing Critical Thinking in Medical Students” reworded as “Applying Case-Based Learning to Improve Critical Thinking in First Year Medical Students” is a better reflection of the abstract’s focus.

Authors, especially in their early career, may not be familiar with the impact of the title, and can benefit from a reviewer’s expertise for recommending a strong title. For example, a weak title may be “New Teaching Strategies for Residents,” while an example of a strong title may be “Enhancing Surgical Resident Learning with Simulation-Based Training: A Practical Guide.”

Tip 2 — Confirm the Abstract Category and Keywords Are Appropriate for the Content

Most educational conferences have preset categories for various submission types. For example, the International Association of Medical Science Educators (IAMSE) Annual Conference has the following session formats: Pre-Conference Faculty Development, Plenary Sessions, Panel Discussions, Focus Sessions, Oral Presentations, and Poster Presentations. The Association of American Medical Colleges (AAMC) Learn Serve Lead Annual Meeting offers the following session formats in its call of medical education proposals: Sessions, Research Abstracts, and Innovation Abstracts. Within these session formats, there might be different subcategories that are offered to further classify the content. Commonly, they are broken down into undergraduate medical education, graduate medical education, faculty development/continuing medical education/continuing professional development, and assessment, and possibly into even further classifications such as basic science, clinical skills, career trajectory, and simulation and technology, to name a few. While it can be a simple mistake for an author to misclassify the submission type and content category for their abstract, it can make a big difference for prospective conference attendees when choosing which session to attend based on job role and interest when perusing the program agenda beforehand; and thus, this is an important aspect for reviewers to tend to. Additionally, as with many journals, some conferences will ask authors to select keywords such as innovation, best practice, or artificial intelligence, which they can include in the conference program. If you have received a submission that does not align to the category for which it was entered, consider contacting the conference planning staff to assign another category. In most instances, as a reviewer, you can provide feedback to the author that a session will be accepted pending it being re-categorized. This can be especially helpful to junior faculty as well as trainees and students who may or may not have attended or submitted to a conference before [18].

Tip 3 — Ensure the Abstract Content Speaks to the Correct Audience

Medical educators and scholars are encouraged to aim for multiple outlets to disseminate our work, or get the most juice out of our squeeze [19]. To do so, it is common for the same educational session to be presented at multiple conferences. While this is recognized as ordinary scholarly practice, it is imperative for authors to make certain the session content is a true match with the mission of the conference. As a reviewer, you are looking to see that all sessions you are appraising speak to the overarching goals and intended audience of the conference. For example, a conference abstract describing a new residency training approach might be adjusted to emphasize its unique implications for faculty development if submitted to a conference where the majority of attendees are faculty.

Tip 4 — Examine if the Background and Rationale for the Session Have Been Relayed Clearly

Most educational abstracts require an introduction or background section to provide information and context to the learning objectives and methods of the session. It is always good practice for medical education scholars to demonstrate their awareness of the existing literature on the topic they are proposing. Reviewers should pay close attention to whether or not the authors present the rationale for the educational session and be prepared to assess the strength of the rationale. This might include supporting literature or current trends that are prompting the need for the session. It also might include an underlying theory or conceptual framework that situates the learning objectives and teaching methods [20, 21]. MedEdMentor provides a list of theories from which authors may find helpful in their scholarly work: https://mededmentor.org/theory-database/ [22]. Providing a theory or conceptual framework may strengthen the credibility and impact of the findings thus allowing reviewers to determine the priority of the abstract in the conference.

Tip 5 — Evaluate the Strength of the Assessment Plan for the Content in the Abstract

The design of educational research often directs the most appropriate way to evaluate participants and/or programs. When applicable, reviewers should see if the authors described their study design. Reviewers should consider if a standardized tool was used, or if a new measurement instrument was exercised. If the latter, reviewers must note if pilot testing for validity was conducted. It is important to also closely examine if the evaluation tool relies solely on learner/participant satisfaction. Such types of evaluations provide limited objectivity and accuracy, and may introduce bias [23]. More so, if the abstract is about a program implementation or new educational training, use of a standardized model, such as Kirkpatrick’s Evaluation Model, to measure results of learning is preferred (Fig. 2) [24].

Fig. 2.

Derived from The New World Kirkpatrick Model, 2016

Tip 6 — Determine Whether Outcomes Are Relevant to the Field

Once the authors have relayed the results of their work or clearly depicted the work in progress, reviewers should evaluate the relevance to the field of medical education. How will this work impact the intended audience/target population? Can the findings be translated into practice? While most research abstracts state the implication of the results in terms of statistical significance with a p value, it is wise to interpret results with respect to magnitude of effect, or the practical implications of a relationship between variables or difference between groups [25]. Furthermore, reviewers must keep in mind that many scholarly efforts in medical education may also demonstrate impact in other ways. These include curricular changes as a result of discovery or the implementation of new institutional policies based on learner feedback or performance [26].

Tip 7 — Assess the Feasibility and Transferability of the Work

Other important components of the results section are the feasibility, sustainability, and transferability of the outcomes. Reviewers should consider whether the educational intervention/study can be replicated again either at the same institution or at other institutions. For example, an online module requiring minimal faculty oversight is more scalable than an intensive, in-person course requiring multiple facilitators. Furthermore, reviewers should try to understand the strengths and deficiencies in the planning and implementation of this work. Reviewers should appraise whether or not the outcomes depicted in the abstract are constructs that will be enduring, meaning they will continue to be relevant to participants in medical education in years to come.

Tip 8 — Appraise the Authors’ Reflective Critique

The final step in Glassick’s criteria is reflective critique [27]. It is essential for reviewers to determine if the authors have engaged in critical reflection after conducting the educational intervention and/or study. Have the authors provided information about what they have learned in directing this initiative and what they would do differently next time? Reflective critique also includes the authors’ explanation of any new opportunities that were identified after completing the work. These opportunities might contain plans for future participants or also plans for the authors’ own professional development. To prompt author reflection, reviewers might pose the following questions in their feedback: “What were the biggest challenges? What could be done differently next time? What future directions does this work suggest?”.

Tip 9 — Exercise Collegiality in Text You Offer as a Reviewer

Author development in medical education varies from each institution [1, 28]. As a reviewer, it is wise to keep in mind that many of the abstracts you may be assigned are likely to be authored by people with varying levels of experience, and that you have met or may work with in the future in the field of medical education. While it is important to be comprehensive, especially if the abstract is awarded a low score, you still need to be conscientious in how you relay constructive feedback. Avoid terms that can be misconstrued as negative or derogatory. For example, instead of, “This abstract is unclear” or “This abstract is poorly written” consider this alternative: “Clarifying the research question and methods would strengthen this abstract.” It can be daunting to submit to a professional conference; therefore, using sensitive yet professional language can foster growth for new authors in medical education while ensuring reviewers are not contributing to sentiments of discouragement.

Tip 10 — Provide Thorough and Specific Feedback

Clear feedback is critical for helping authors learn how to write a better abstract. If you give authors corrective feedback that may be difficult to digest, it is important to also provide clear ways that the authors can improve. Questions reviewers should ask themselves are the following: “Is the feedback helpful?” “How might I feel if I received this feedback?” “How can it be written to maximize its effectiveness?” In the truest spirit of the review process, reviewers of medical education conference abstracts may take these opportunities to serve as peer leaders or mentors for those authors who need guidance and support. Good feedback should be actionable, objective, based on explicit criteria, concise and specific, allow for future improvement, grounded in truth and evidence, and focused on what/how. The best feedback, whether it is corrective or reinforcing, is specific with concrete suggestions [29]. To illustrate this, if a conference abstract was lacking in detail about the methods section, rather than a reviewer stating “insufficient detail,” a preferred manner of giving feedback might be, “The methods section would be more comprehensive with information about the sampling strategy, psychometric properties of the standardized tool used, and a step-by-step plan of how data were analyzed.”

Tip 11 — Exhibit Awareness of Character Count Limitations for the Abstract

Abstracts, by nature, must communicate a lot of information in a small package. Reviewers need to be familiar with character and word count limits that authors must abide by in the abstract submission instructions. Authors may find it difficult to elaborate on elements in the abstract because of these limits. Reviewers need to remain cognizant of this and realize that abbreviated explanations may be the result of these administrative constraints, and not by negligence of the authors. Reviewers might suggest that further description of such elements of the abstract be provided in the actual conference presentation, if accepted.

Tip 12 — Acknowledge the Challenges You Are Facing as the Reviewer

Reviewers face additional challenges while reviewing abstracts that they need to be aware of, including time pressures, bias, and a lack of feedback on the reviews they submit. First, there is often a quick turnaround between receipt of abstracts and when reviews are due, and it is not uncommon for reviewers to receive 10 or more abstracts to review. Coupled with day-to-day responsibilities, this leads to immense time pressure on reviewers. Further, it can be challenging for reviewers to prioritize this task over other commitments, especially as reviewing is often “unpaid” [30], or takes place outside of traditional working hours. This time pressure can result in random or inconsistent reviewing, potentially leading to a “success catastrophe” where conferences become overwhelmed with submissions [31].

Unconscious bias can also create a challenge for reviewers in the peer review process and this is often less recognized in the literature compared to any potential bias in the abstract material presented by the authors [32]. These biases can stem from many items, including personal attributes of authors and reviewers, including gender, institutional affiliation, and regional factors [33]. The peer review process can also be influenced by intellectual bias, which may be difficult to eliminate even with disclosure of conflicts of interest. Other forms of bias include content-based bias, confirmation bias, and bias against interdisciplinary research. On a broader scale, double-blind or open peer review processes can help, along with utilizing a diverse group of reviewers and being transparent about review practices [32].

Finally, reviewers typically do not receive feedback on their reviews, which means that there is little chance to learn and grow in their skillset as reviewers. This is especially challenging for new reviewers. To resolve this challenge, it is important for reviewers to seek out mentorship and guidance on their comments and assessments of abstracts from more experienced colleagues. Additional reviewer training through professional organizations can also be helpful. Of note, the authors have all created and conducted training on conference abstract reviewing. Nonaillada, J. and Holterman, L.A. taught a webinar sponsored by the Northeast Group on Educational Affairs (NEGEA) in the fall of 2023: “Medical Education Abstract Reviews: A Go To Guide for Reviewers Providing Effective Feedback and Fostering Critical Thinking,” and at the time of this writing, this webinar is publicly available at https://vimeo.com/showcase/10543683?page=1.

Berry, A. created a module “Tips and Tricks for Reviewing Conference Abstracts” for the Medical Education Scholarship Research and Evaluation (MESRE) of the AAMC Group on Educational Affairs, and at the time of this writing, this resource is publicly available at https://rise.articulate.com/share/5wwac2qV-qthP8n5rnGZIe7FTI7hoED9#/.

Further guidance from the Committee on Publication Ethics which can be applied across all disciplines might be useful [34], as well as the Certified Peer Reviewer Course from Elsevier Research Academy https://researcheracademy.elsevier.com/navigating-peer-review/certified-peer-reviewer-course.

Conclusions

With all these recommendations, the review process for medical education conference abstracts can be overwhelming, especially for new reviewers. Through our own experience, research, and professional development, we have found the above tips to be highly useful in navigating this important task. We believe this handy guide, honed through presentations and workshops, will help reviewers approach this responsibility with confidence and positivity (Appendix 1). We want to recommend our audience of readers to engage in active reflection on their own strengths in reviewing medical education conference abstracts, while charging them to take action on areas they might be able to improve upon based on the 12 tips we have put forth. We look forward to your future reviews for the advancement of scholarship in medical education.

Appendix 1. Checklist for Medical Education Conference Abstract Reviewers

1. Does the title capture the essence of the abstract?

2. Is the abstract category/submission type and keywords accurate?

3. Is the abstract appropriate for the typical audience of this conference?

4. Have the authors included background information supporting the rationale for this abstract?

5. What is the strength of the assessment plan or measure of success in this abstract?

6. Are the outcomes proposed by the authors relevant to medical education?

7. Is the intervention or innovation described by the authors feasible and transferable to other institutions?

8. Have the authors engaged in their own reflective critique of their work? If not, how can I prompt them to do so?

9. Have I stated my review in a collegial tone?

10. Have I provided feedback that is specific and actionable with examples of next steps the author can take?

11. Am I knowledgeable of the character count limitations that the authors must abide by?

12. Have I dedicated enough time to adequately complete this review where I can be free of any time pressure? Have I approached this review with an open mind regardless of my personal traits and how I perform my own work in medical education? Do I know who to contact from the conference to inquire about feedback on my abstract reviews?

Author Contribution

All authors contributed to the paper conception and design. All authors read and approved the drafts and final manuscript.

Data Availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Declarations

Conflict of Interest

The authors have no competing interests to declare that are relevant to the content of this article.