Up for Debate: Re-Envisioning the Journal Club with an Article-Centric Student Debate

Abstract

Effective utilization of evidence-based medicine requires skillful development of a critical literature evaluation process. Although traditional journal club activities are a common modality to teach and refine these skills, they may limit a learner's motivation to perform a well-rounded critique of primary literature.

Innovation:

In response to the challenges with these traditional formats, we describe a novel approach to refining critical literature evaluation skills in an Advanced Pharmacy Practice Experience (APPE) setting utilizing a focused, article-centric journal club debate. Students, in pairs, are assigned a single article and are tasked with building critical arguments for both pro and con sides of the article, which culminates in a one-on-one debate.

Key Findings:

The debate has been well received by students and faculty for increasing engagement in the critical literature evaluation process. The article-centric nature of the debate pushes students to a deeper understanding of an article's merits and pitfalls. Ongoing limitations include significant faculty assessment burden and lack of a standardized, debate-specific evaluation rubric.

Next Steps:

Future efforts should focus on evaluating student performance and perceptions of the debate compared to traditional journal club formats utilizing pre- and post- surveys. The creation of a debate-specific grading rubric may streamline the evaluation process and reduce faculty assessment burden.

Description of the Problem

Evidence-based medicine is an expectation and cornerstone of contemporary pharmacy practice, regardless of the patient care setting. The need for skill development in primary literature evaluation is highlighted across the current accreditation standards from the Accreditation Counsel for Pharmacy Education (ACPE) 2016. Specifically, the Standards require that practice-ready graduates be able to prioritize, collect, and interpret evidence, then use clinical reasoning to apply evidence to patient care (ACPE Standard 2.1 and 25.7).¹ Activities intended to bolster these skills are built longitudinally throughout pharmacy curricula, particularly in the experiential setting. Journal clubs are frequently utilized to give students practical repetition at critically analyzing, evaluating, synthesizing, and applying primary literature to patient care.

Challenges associated with meaningful journal club activities are documented in the literature and serve as an impetus to create new strategies to improve critical literature evaluation. Students often perform superficial critical analyses by mimicking the strengths, limitations, and critical discussion presented by the authors themselves.² Students also tend to focus on one side of a given argument and fail to perform a well-rounded appraisal that identifies an article's merits while also acknowledging its pitfalls.

Many faculty and preceptors, authors included, require Advanced Pharmacy Practice Experience (APPE) students to complete journal club presentations using a traditional format on their respective rotations. More recently, however, we have re-envisioned our journal club presentations into a debate-style format in response to student struggles and our experiences with the superficial depth of traditional journal club formats. The debate format was chosen and implemented specifically to foster the development of well-rounded understandings and thorough critical analysis of current literature.

Innovation

A Case for Debates

The use of Lincoln-Douglas style debates as a teaching tool is a known concept in the educational landscape. In pharmacy curricula, debates have been successfully implemented in both the didactic and experiential settings; however, evidence in the experiential setting is sparse.^3–5

To the best of our knowledge, there are only two reports supporting implementation of a debate activity into pharmacy experiential education, and only one of which explicitly developed the debate as a novel format for a journal club, specifically.^4,5 In both examples, students explore literature analysis in the context of a clinical controversy by being assigned to debate a specific position within a clinical context. Toor et al. found that students perceived themselves to have more confidence in finding, comparing, and retaining relevant information from primary literature following the activity.⁵ Similar improvements in student confidence were seen by DyBoarman et al.⁴ In both instances, student satisfaction with the activity was high, and many were pleased with the development of their skills as a result of the activity.

The Journal Club Debate

The journal club debate is incorporated as a required element of the authors’ respective five-week acute care APPEs; however, it could be easily implemented into other APPE types. Conceptually, the debate is very simple: two students are pitted against each other to debate the application of a recently published journal article relevant to pharmacotherapy. Articles are carefully chosen by faculty, and the two sides of the debate are explicitly defined for the students, one as the “pro” and the other the “con.” Articles are typically newly selected each rotation but may be recycled if neither student has been exposed to the article in question. Examples of suggested debate articles are detailed in Table 1.

Table 1. Examples of Suggested Debate Articles and Rationale.

Article	Rationale for Debate
McMurray JJ, Packer M, Desai AS, et al. Angiotensin-neprilysin inhibition versus enalapril in heart failure (PARADIGM-HF). N Engl J Med. 2014;371 (11):993–1004.	New therapeutic option versus standard of care. Pro: Argues for the new neprilysin combination therapy in heart failure Con: Argues for standard guideline-directed heart failure therapy
Bonaca MP, Braunwald E, Sabatine MS, et al. Long-term use of ticagrelor in patients with prior myocardial infarction (PEGASUS). N Engl J Med. 2015;373 (13):1274–1275.	New treatment strategy versus standard of care Pro: Argues for extended duration ticagrelor Con: Argues for standard duration ticagrelor
Lazarus B, Chen Y, Wilson FP, et al. Proton pump inhibitor use and the risk of chronic kidney disease. JAMA Intern Med. 2016;176 (2):238–246.	Cohort-based adverse effect evaluation Pro: Argues for increased risk of CKD with PPIs Con: Argues against increased of CKD with PPIs

Students complete a two-part assignment: a one-page overview of objective aspects of the article and a PowerPoint presentation that presents both pro and con sides. Students may work collaboratively on the overview, which mimics components of a traditional journal club presentation. This portion of the assignment focuses only on the objective information from the trial, with no critique on the appropriateness of any aspect of the trial. In the second part of the assignment, each student develops a PowerPoint presentation addressing three major and distinct arguments supporting each side, pro and con, along with closing statements. Students are instructed to include both efficacy and safety implications in their arguments as well as article strengths, weaknesses, and application to real-world patient care. Evidence-based support for each argument must come from reputable sources, including but not limited to other pertinent primary literature. It should be noted that students must prepare a case for both sides of the debate, as their position is determined immediately prior to the debate presentation.

The timeline of the activity begins during rotation orientation where students receive the assignment instructions, the assigned article, and the construct of each side (pro and con). This occurs typically 2-3 weeks before the debate. Students must develop a list of references to support each side during the debate. Four days prior to the debate presentation, the final reference list is submitted and shared with the other student. No other resources are allowed to be used, which allows each student to develop his or her arguments or prepare for rebuttals. The day prior to the debate, all other materials are due to the preceptor, including slides and the overview handout.

To start the debate, the two students are assigned to present either the pro or con side of the article using coin-flip randomization. Students collaboratively present the trial overview, after which the debate begins. Table 2 depicts the presentation schedule with alternating arguments, rebuttals, and specified time allotments for each aspect of the debate. Arguments and rebuttals are kept to a strict time ceiling. Students are permitted to use notes to aid in the delivery of their arguments; however, additional time is not allotted for students to sort through them. Following the presentation of all arguments and closing statements, there is a brief questionand-answer period from student and faculty attendees. Students are individually evaluated by faculty using a standardized journal club presentation rubric for all APPE rotations within our school of pharmacy. This rubric assesses domains such as trial overview, critical evaluation, clinical application, organization, ability to answer questions, quality of audio-visual materials, and presentation skills. Feedback is provided to students as part of the summative rotation evaluation in which the debate is a component of their final APPE grade.

Table 2. Debate Presentation Schedule and Time Allocation.

Presentation	Item Time
Student A and B- Study Overview	10 min
Student A- Position #1	3 min
Student B- Rebuttal	4 min
Student B- Position #1	3 min
Student A- Rebuttal	4 min
Student A- Position #2	3 min
Student B- Rebuttal	4 min
Student B- Position #2	3 min
Student A- Rebuttal	4 min
Student A- Position #3	3 min
Student B- Rebuttal	4 min
Student B- Position #3	3 min
Student A- Rebuttal	4 min
Student A Closing statement	4 min
Student B Closing statement.	4 min
Audience Q/A	10 min
Total Debate Time	70 min

Critical Analysis

Lessons Learned

Since its introduction into our APPE rotation in May 2016, approximately 58 students have participated in the journal club debate. Student feedback about the debate, both formal and informal, has been overwhelmingly positive. Students have enjoyed the debate as a unique and increasingly more detailed approach to critical literature analysis with a more enjoyable methodology. Observations from faculty have paralleled many of the positive aspects of the student feedback. As compared to traditional journal club formats, students seem to display a greater breadth and depth in evaluation of study strengths and weaknesses. This may arise from the activity's requirement that students create arguments for both sides of the study. This pushes students to perform a focused evaluation of the positive and negative aspects of the study design and methodology to fortify their individual arguments. Presenting and defending their arguments to an opposing student inherently motivates students to have a more well developed and supported understanding of the study and its implications. Similarly, students have more effectively used supporting primary literature to complement their arguments and rebuttals in the debate. Utilizing supporting data from outside primary sources has advanced student understanding of contextual application of the literature within the current clinical landscape.

Despite anecdotal benefits, some aspects of the debate have needed continual improvement to optimize the learning experience. As a result, the journal club debate has evolved through multiple iterations based on faculty observations and student feedback to curtail areas of limitation. One aspect identified was the time allotted for arguments and rebuttals. Initially, the timeslots were equal; however, given the impromptu nature of the rebuttal, the time was extended to allow students to collect their thoughts and develop appropriate counter points. A second area of revision dealt with how students utilized outside resources in their debate preparations. During initial versions of the debate, students were often unfamiliar with other supporting resources used by the opposing student. Requiring student pairs to submit their list of references to each other several days in advance enabled them to prepare rebuttals for both sides of the debate. These changes to time allotments and reference submission processes increased the quality of rebuttals.

While minor changes have helped improve aspects of the debate, a few challenges remain. One such pitfall is student collaboration on preparation of arguments and rebuttals. This has been evident through “canned” rebuttals and a seamless flow of point to counter point, which can detract from the purpose of the assignment. The inability to fully police any collaborative effort is an issue that may warrant further investigation. Another area that appears to be a problematic is the students’ ability to fully utilize their time allotment. More often than not, students leave extra time at the end of each portion of the debate that could otherwise be utilized to further develop and explain why their argument is important to the interpretation and application of the article. Similarly, students have some difficulty creating diverse arguments apart from the results of the study. Often the pro student will discuss the efficacy and the con student the safety components of the results with little application. However, even in cases of sub-optimal critique, students’ application of the strengths and weaknesses far exceed previous observations within a traditional journal club format. A continual focus on adequate orientation, evaluation, and revision of the debate is important to minimize these limitations.

Beyond these student-focused challenges, there are also some limitations from the faculty perspective. The total time to complete the debate is likely longer than a traditional journal club. The debate portion lengthens the time spent discussing strengths and weaknesses, which may improve student understanding but also requires a larger time commitment for graders. The back and forth nature of the debate can make it cumbersome to quickly evaluate the slides, critical evaluation skills, and presentation ability for each student individually. As a result, additional time may be needed to comb through presentation materials and fully evaluate all aspects of student performance. One area of future exploration is the development or adaptation of an objective assessment tool to formally assess the learning activity. A summative rubric-based evaluation has been helpful in the early success of the debate; however, this rubric is not specific to a debate format. A more focused and comparative evaluation tool should be created to more fully identify the ability of the assignment to improve critical literature analysis, while also minimizing assessment burden on graders.

This unique, article-centric debate has brought a higher level of student engagement in the critical literature evaluation process. It continues to evolve in efforts to enhance the student experience while streamlining faculty workload.

Next Steps

Future efforts should focus on evaluating student performance, engagement, and perceptions of the debate compared to traditional journal club formats. As this technique is adapted in other settings, a few recommendations may be pertinent to allow for the early success of learners. First, journal article selection is paramount. The article chosen should have controversial conclusions and no definitive answer to the clinical question it set out to answer. Failing to do so may reduce the efficacy of the experience. Second, providing clear and timely instructions well in advance is critical. Given the novelty of the debate, students may become confused on the framework and flow. Detailed instructions on expectations, debate etiquette, logistics, and processes (Table 2) may help students visualize the assignment and prevent confusion during debate preparation and presentation. With a well-developed assignment timeline, detailed orientation, and purposeful article selection, a journal club debate can add a refreshing twist to traditional formats that engages not only the debating students, but also the audience and faculty as well. So what do you say, are you up for a debate?