Planning and executing orthopedic journal clubs

The earliest reference to a journal club is possibly in the biography of Sir James Paget during the period of 1835-1854.¹ It was noted that a group of students met in a room over a baker's shop near St. Bartholomew's Hospital to read journals or play cards. However, Sir William Osler was credited to have conducted the first journal club on record in 1875 in North America.¹,² Its purpose was “for the purchase and distribution of periodicals to which he (Osler) could ill afford to subscribe as an individual”.¹ Osler then transferred to Johns Hopkins University Medical School in 1889 where he established the Book and Journal Club. There, over dinner, the members recommended new books for the library and reviewed the latest in medical literature.³ Journal clubs have now become ubiquitous in graduate medical education training programs, including orthopedic surgery.⁴ Greene surveyed 149 North American orthopedic surgery training programs and found that 99% included a regularly scheduled journal club.⁴ They initially served to help practitioners keep up to date with the latest published literature.¹,^5–7 More recently, however, they have been used to teach critical appraisal skills, research design, medical statistics, clinical decision theory, clinical epidemiology and encourage “evidence-based” medical practices.⁸

The purpose of this article is to provide organizers with tools to plan and execute successful journal clubs in orthopedic surgery training programs. Guyatt noted that not all clinicians need to appraise evidence from scratch, but all need some skills.⁹ Thus, each journal club needs to respond to the needs of its participants and thus no single ideal format exists.

PURPOSE AND GOALS OF JOURNAL CLUBS

Originally, journal clubs were established for keeping abreast with the current literature. This has become an unrealistic goal with the explosion of orthopedic journals. Current goals include promoting evidence-based orthopedics and learning general critical appraisal skills for residents. Evidence-based medicine (EBM) is becoming an accepted educational paradigm in medical education.¹⁰,¹¹ EBM relies on the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.¹² There are four key elements of EBM: question setting, literature searching, selecting relevant papers and critical appraisal. Historically, internal medicine training programs in Canada and the United States have included the use of EBM practices. Recently, surgical training programs have started to include these principles into their curricula. Evidence-based surgery is often poorly incorporated into orthopedic residency clinical training. Journal clubs can be an ideal vehicle for evidence-based orthopedic surgery education.

Several articles have studied the importance of teaching critical appraisal skills to surgical residents.⁴,^13–15 This goal was ranked the most important in Greene's survey of 149 program directors of North American orthopedic training programs.⁴ Other important goals identified in his study were instilling the habit of reading scientific journals (20% of programs) and facilitating residents learning about current research (9% of programs).

Journal clubs in general surgery were surveyed with questionnaires filled out by program directors in American institutions.¹⁴ A response rate of 80% from 278 programs identified that 64% of directors thought their journal club was important or very important to the educational mission of their training program. Other top purposes included learning literature review skills (87.5% of directors), providing training in research education (52.9%) and research design (52.4%).

Clinical knowledge transmission was found to be a more important goal in journal clubs in orthopedic fellowship programs over residency programs. This was found in a survey of 57 hand surgery fellowship directors, where the current literature was reviewed for the fellows and faculty as a primary goal.¹⁵ These fellowship directors felt this journal club was very important for fellowship training.

Secondary goals of journal clubs for residents include increasing reading comprehension, fostering resident-faculty relations and learning orthopedics in an alternative manner.⁴,¹⁴

In American internal medicine residency programs, national accreditation requirements mandate that residents participate in journal clubs regarding the critical assessment of the literature, clinical epidemiology and medical statistics. This may become mandatory in orthopedic surgery training programs in the future. In addition, faculty members can fulfill their continuing education requirements if mandated by their national colleges. National qualifying examinations in some countries for some specialties may include critical appraisal of a paper, as seen in psychiatry.¹⁶

The target audience must be identified, including their needs and interests. This will influence the format of the journal club and papers chosen. Emphasis may be placed on keeping up to date with recent advances in orthopedics versus the educational value of critical appraisal of papers. These goals are not mutually exclusive, but each requires significant time and organizational commitments.

PREVALENCE AND CHARACTERISTICS OF ORTHOPEDIC JOURNAL CLUBS

There are only two known published papers regarding orthopedic journal clubs. In 2000, Greene was the first to survey 161 North American orthopedic training program chairmen.⁴ One hundred and forty-nine programs (93%) responded. Regularly scheduled journal clubs were part of the educational program in 99% of the orthopedic residency training programs. Seventy-eight per cent met once per month and 9% met twice per month. Papers from more than one scientific journal were reviewed in 82% of clubs. Half of the meetings (50%) restricted discussion to scientific studies only. A minority of clubs discussed case reports or review articles. The average number of assigned papers to each resident was 2.6 (range: 1-8). A designated faculty member (38%) or the chairman of the orthopedic program (17%) moderated the discussion, whereas 32% were moderated by rotating faculty and 13% by chief residents. Half of the meetings were held at the departmental office, 31% met at the home of a faculty member and 18% met at a restaurant. The meetings were held in the evening (68%) or at the start of the work day (29%). All journal clubs met for a period of one to two hours. Attendance by the residents at the club was rated as 80% to 100% by 59% of programs, 60% to 80% by 36% of programs and 40% to 60% by 4% of programs. A full meal was provided at 50% of the meetings or a light snack provided at 37%, mostly funded by departmental funds. The top three goals of these clubs were: teaching residents how to evaluate scientific articles, instilling a habit of reading scientific journals and facilitating residents learning about current research.

Dirschl, Tornetta and Bhandari published a comprehensive review of orthopedic journal clubs in 2003.¹⁷ These authors discussed methods to design, conduct and evaluate these educational meetings with current evidence. The following discussion will address these issues.

PLANNING AND ORGANIZATIONAL ISSUES

Not all journal clubs are successful despite their presence in the majority of the orthopedic training programs. Sidorov⁸ defined successful journal clubs as those that meet educational objectives and as those that continually promote and maintain resident interest. Jones et al suggested that all departments with a journal club should regularly revise their selection of journals in order to increase the value of this important educational process.¹⁸

LEADERSHIP

The responsibility of running the journal club by a designated individual or group of individuals is important. Effective journal clubs are correlated with having a designated leader.⁵ An interest in medical education and a belief that the journal club is important are essential attributes of the leader and facilitators. This person or persons would be responsible for the organization, execution and evaluation of the club. In addition, having residents actively involved in planning and operating the club has been found to be important.¹⁹,²⁰ Having a skilled moderator, whether resident or faculty member, is important to the value and attendance of the club.¹⁷

Part of the success of any journal club is the organization within which it is run. If the orthopedic division or department supports this activity, then it is more likely to be successful.

LEARNING OBJECTIVES

Although many studies state educational goals for the journal club, many programs do not have formal written learning objectives. This was demonstrated in a survey of emergency medicine program directors, where 42% of programs had not established objectives.¹⁹ These objectives are important to formalize as they articulate and communicate the goals of the club. Parameters of the meeting can be defined regarding the content, number of papers to be reviewed, which journals' papers should be obtained from, the format and setting. The effectiveness of the club can then be measured based on these objectives.¹⁷

ENVIRONMENT

Maintaining boundaries is important to the success of the club. Thus trainees should be relieved from clinical duties, except emergency patient care responsibilities, especially with support from those faculty members not attending the journal club. This aids the session to start and stop on time. Papers should be distributed well in advance to attendees and the room should be comfortable. By maintaining clear boundaries, group theory suggests that the environment allows a sense of security and thus allows creative thinking.¹⁶

Audience participation is crucial to the success of journal clubs, unlike traditional teaching conferences. It has been shown that interventions involving only didactic sessions do not change physician behavior.²¹ The educational value of the meeting is optimized by the interactions of the attendees and the dynamic exchange of opinions and ideas.¹⁷

Mandatory attendance has been found to be important⁸ with respect to high attendance rates and longevity of journal clubs. Continuity is promoted by having a regular group of attendees. On the other hand, trainees may feel attendance is imposed on them rather than viewing the club as an educational opportunity. This may be addressed by allowing residents to set their own rules such as journal club presenter opportunities and participating in choosing papers to discuss.

The venue should suit the group. It should facilitate relaxation and conversation. Promoting eye contact and encouraging active participation are facilitated by seating individuals in a circle or a horseshoe.¹⁶,¹⁷ Options for meeting locations include departmental offices or conference rooms, faculty homes or restaurants.⁴ Restaurants or pubs may be distracting if there is significant ambient noise or music drowning out the discussion of papers.

Optimal discussion groups are composed of 10 to 12 members.¹⁷ Thus in large journal clubs, it may be necessary to divide attendees into these smaller groups to facilitate participation and discussion amongst all members.

MODERATORS

Faculty support with independent running of the journal club (by a senior trainee) was shown to lead to high attendance rates.⁸,²⁰ Having a trainee as a club leader may make the fellow trainees feel less intimidated in asking questions. Senior trainees, rather than the faculty, may be more attuned to the needs of their junior colleagues.

FREQUENCY

Monthly journal clubs are the norm for the majority of postgraduate training programs.⁴,²² More frequent meetings may be difficult, especially if residents are expected to review numerous papers for each club or if a detailed critical appraisal is required.

TIME

Residents and faculty are busy attending to clinical duties during daytime working hours in the majority of orthopaedic training programs. Thus having journal clubs early in the morning before the work day or in the evenings is more convenient for all involved.¹⁷ In contrast, midday meetings may be more convenient in nonsurgical training programs.

FOOD

The regular provision of food has been associated with successful journal clubs.⁴,²² In orthopedics, 87% of meetings provide some sort of food.⁴ Longevity and high attendance rates are correlated with the provision of food.⁸ Funding journal club meals may be provided by individual faculty or by divisional or departmental support. There have been recent strict North American guidelines on industry funding of medical educational events. This same issue has been discussed in the United Kingdom.³⁹

NUMBER OF PAPERS TO REVIEW

There is significant variability in the number of papers to review in journal clubs.¹⁵,²³,²⁴ A range of three to 10 papers seems to be a reasonable trade-off between having too many articles (limiting valuable discussion) and too few articles (limiting the breadth of information).¹⁷ The number of papers and depth of review should be individualized by each training program.

LINKING JOURNAL CLUB TO THE CURRICULUM

As mentioned earlier, journal clubs can be a venue to teach critical appraisal skills to residents, either as formal sessions or through discussion of the chosen papers. Supplemental formal lectures on this topic could also be provided in the core curriculum of training programs. This has been an important stated goal of orthopedic program directors.⁴ Residents can also be evaluated in journal club sessions regarding national college training requirements, such as the core competencies from The Royal College of Physicians and Surgeons of Canada Canmeds program or The American Council for Graduate Medical Education. If this is an important goal for program directors, establishing formal objectives and evaluation of the journal club would aid in an external review of the training program by their respective national college.

Many authors have studied the educational effects of journal clubs in residency training programs. In one study, residents whose meetings focused on epidemiology and biostatistics thought they read with more attention to the study design and methodology, in self-assessment, than the control group of residents in a traditional journal club.²⁰ In this randomized study, there were no significant differences between the two groups of residents in objective testing of knowledge of epidemiology, biostatistics, nor critical evaluation.

In a similar study of emergency medicine residents with a prospective, case-controlled design, the study group (journal club with emphasis on epidemiology principles) and control group (traditional journal club) of residents improved their performance over the one-year period.²⁵ No difference was shown between groups with objective pre-testing and post-testing in gaining knowledge in clinical epidemiology principles. This similar result was found in another study of two pediatric training programs, whose only difference was that one program had two introductory lectures on epidemiology principles.²⁶

IMPACTS OF JOURNAL CLUB ON READING HABITS, KNOWLEDGE AND SKILLS

Parkes et al revealed a small evidence base that journal clubs probably do improve knowledge in biostatistics and clinical epidemiology in a Cochrane review.²⁷ Conflicting evidence exists on whether they enhance critical appraisal skills. Self-assessment shows more improvement than objective measurement of skills.²² The impact of journal clubs on patient outcomes has not yet been investigated in a well-designed trial, as there is difficulty in measuring this process.

STRUCTURED REVIEW INSTRUMENTS

Prepared checklists exist to review papers in a structured format. These can be found in reading guides such as the Users' Guide to Medical and Surgical Literature series of articles in the Journal of Bone and Joint Surgery,²⁸ textbooks²⁹ and the McMaster reading guides (Department of Clinical Epidemiology and Biostatistics, McMaster University, 1981). The advantage is that the reader can distinguish between proper and poor methodology in a more consistent manner.

Guidelines and dataforms for critical appraisal of published articles may be downloaded from the Oxford University evidence-based medicine website at http://www.cebm.net/downloads.asp.

Alguire et al²² published a packaged critical appraisal curriculum that included reading guides and other teaching materials.

The use of a reading checklist was evaluated in an emergency medicine training program.²⁵ Resident use of the structured review instrument was associated with higher overall satisfaction with the journal club format and a perceived improvement in clinical education. Improvement in critical appraisal skills was not tested. The residents did not perceive an increased workload, nor did attendance rates decrease.

Thus the use of a structured review instrument is an effective adjunct to the journal club for teaching and applying critical appraisal. This tool may be used for resident evaluation of knowledge acquisition and the impact of the meeting on the overall educational training program.

TYPES OF JOURNAL CLUBS

Many types of journal clubs exist. No one type is superior to the others. This allows organizers to tailor the club to their educational objectives. An individual residency training program may choose to adopt more than one type, as each one may not address all goals and objectives desired. Innovative types of journal clubs used elsewhere may generate ideas for local use.

EVIDENCE-BASED JOURNAL CLUBS

EBM relies on the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.¹² There are four key elements of EBM: question setting, literature searching, selecting relevant papers and critical appraisal. Thus EBM journal clubs are a more extensive undertaking, with critical appraisal being just one key element. Thus a wider spectrum of skills is required.

Question setting can start with a real life clinical scenario. These scenarios may come from the outpatient clinic, emergency room or the operating room.

Literature searching is the next step in this format. If no one is confident in this skill, local librarians may help in finding the relevant papers for a topic. Knowledge of search strategies and filtering techniques are important to select relevant articles. Interestingly, in a study by Coomaraswamy et al³⁰ of journal clubs in obstetrics and gynecology, 11 of 55 consecutive appraisals missed the article most relevant to the clinical question. Critical appraisal then follows with a structured review instrument. Critical appraisal is the systematic weighing up of evidence. The principles of research methods and statistics are the focus.

There are several possible formats to run the EBM journal club. One session may focus on question setting and identifying the most relevant papers. The next session can appraise them. Secondly, attendees can prepare the first three steps in advance, leaving the club's time for critical appraisal. Lastly, a question could be posed for a debate between two teams. A chairperson could mediate arguments from each side.

An evidence-based journal club has been designed for faculty interested in medical education, which can be applied to almost any clinical topic or specialty area.⁸ Here, unread journals relating to a specific topic area are brought to the conference. The attendees read the titles of the papers out loud in small groups and ask three questions to evaluate if specific papers achieve relevancy criteria: 1. Does the article have an impact on the topic? 2. Does the article focus on a common issue? 3. Will the article change practice? The article is tagged for critical review if the answer is yes to all three questions. It is then assigned to two attendees for review and discussion at the next meeting. Challenging the participants to think critically about which article to read was linked to high satisfaction levels.

In an obstetrics and gynecology program, a very structured 12-month evidence-based curriculum has been designed to cover 24 topics in epidemiology, biostatistics and experimental design.³¹ The journal club met monthly for two hours and two sets of papers were distributed for each session. The first set consisted of literature on topics in epidemiology, biostatistics and experimental design, with two to six papers distributed for each session. The second set of papers consisted of articles for critical review, selected from all medical specialties to illustrate the concepts learned from the first set of papers for each session. It was reported as well received by the residents, although a great deal of preparation was necessary. The residents commented that they would have preferred all of the papers to have come from the obstetrical and gynecological literature instead of from general medicine. The full list of papers can be reviewed in this report.³¹

Most EBM occurs in a journal club format.¹¹ Training programs may offer a seminar series, where residents present cases, generate questions and critically appraise the literature. Simpler concepts of EBM may be assigned to junior level residents, such as therapy and diagnosis. More complex topics, such as meta-analysis, decision analysis and economic analysis, may be assigned to senior level residents.

It should be noted that not all residents or faculty are interested in attaining an advanced level of EBM skills.⁹ Thus other sources of papers may include the use of evidence-based summaries written by others, such as the evidence-based orthopedics section of the Journal of Bone and Joint Surgery. Learning a basic subset of skills will help in training the next generation of orthopedic surgeons to be up-to-date practitioners, who are referred to as evidence-based users.

JOURNAL CLUBS BASED ON CRITICAL APPRAISAL

To reiterate, critical appraisal is the systematic weighing up of evidence. The principles of research methods and statistics are the focus of this format. A structured review instrument is used. Up to five papers are reviewed at each meeting. Classic papers may be reviewed, which are frequently referred to in the literature. Attendees can then assess the validity of the conclusions drawn.

By dividing attendees into small groups of 10 to 12 members, more audience participation may occur.¹²,¹⁷ Each group is assigned critical appraisal questions after the presenter briefly introduces the paper. The session is completed with a general group discussion of the correct answers. The Pocket Guide to Critical Appraisal³² may aid in selecting appropriate questions.

One disadvantage of this format is that attendees, including faculty members, may feel intimidated to participate if they don't feel confident with their critical appraisal skills.

Inviting a statistician to the journal club will aid in understanding statistical issues.¹⁶ An article from internal medicine³³ reported such a format. One resident volunteers to select and present one clinical study for the journal club. All residents are provided with the paper two weeks before the meeting. At the same time, the resident and a faculty mentor devise and provide a series of questions to all participants regarding the quality of the research, appropriateness of the data and methods and the validity of the conclusions. At the journal club, discussion focuses on answering the questions rather than the clinical aspects of the paper. A great deal of faculty involvement and facilitation is required to succeed in this format.

JOURNAL CLUBS BASED ON KNOWLEDGE TRANSMISSION

This may be the most popular form of journal club in most training programs. Despite many formats that exist, the main goal is to provide a brief overview of new developments in a topic area or subspecialty.

A survey of orthopedic hand surgery fellowship programs revealed that as many as 12 papers are discussed in a one-hour conference.¹⁵ The most important goal of these meetings is to review the current literature in hand surgery. A slight modification of this format is to have each attendee review one paper from a journal or topic area.¹⁷ Each attendee then presents to the group the key contributions of the paper to the topic or specialty area.

THE NO PREPARATION JOURNAL CLUB

In this unique format, the paper(s) for discussion is not distributed to the attendees prior to the journal club.²⁴ Here, a resident chooses a paper and then discusses it at the meeting with the aid of a faculty moderator. The attendees do not read the paper beforehand. A brief presentation of the research question of the selected study begins the meeting. The participants are then asked to suggest appropriate study designs to address the research question, followed by the author's chosen study method. The attendees are then asked to assume the author's chosen study design and suggest and discuss additional details of study methodology. This process continues until all relevant aspects of the study methodology are discussed. The results and conclusions are then reviewed. It was reported in this article that more positive discussion of methodological issues occurred rather than participants being hypercritical, which occurs when papers are reviewed before the journal club meeting. As well, resident workloads were decreased by not having to prepare in advance of the meeting, attendance rates were higher and residents gained a greater appreciation of the author's design choices.

INTERNET-BASED JOURNAL CLUBS

An otolaryngology residency program reported on a journal club employing e-mail and the internet to archive current literature reviews in otolaryngology.³⁴ Each training year in the residency program (PGY2, PGY3), was assigned a particular journal to review. The residents in each year divide the articles in their assigned journal and each resident prepares written reviews of the article(s), usually one to five papers per month. Each resident then submits their review(s) via e-mail to all residents and faculty in the department. Thus a complete review of the current month's otolaryngology literature is provided. The best four to 10 articles are then selected to discuss at that month's journal club. These monthly reviews are archived on a server, which are accessible from the departmental website to all residents and faculty. It can also be searched by title, date, journal and thread.

EVALUATION OF THE JOURNAL CLUB

Regardless of the format(s) chosen, periodic restructuring of the club will be necessary to meet new goals, as is done for all educational activities. This will help to maintain enthusiasm and effectiveness. To evaluate the effectiveness on individuals' learning, Morrison et al³⁵ have developed a list of appropriate questions.

Periodically asking the attendees about their satisfaction with the club is one way to modify the meeting if needed. Anonymous feedback from the residents may alert organizers about strengths and weaknesses and potentials for improvement.¹⁷ The residents could also be asked to self-assess their clinical reading habits and critical appraisal skills. Objective measures of understanding critical appraisal skills could be performed to document knowledge gain in epidemiologic principles, with a pre-test and post-test format²⁴ or by the evaluation of a factitious standardized article.²³,²⁵ These periodic evaluation methods will aid in improving the educational experience of the journal club, regardless of its format.

CATALOGUING CRITICALLY APPRAISED ARTICLES FROM JOURNAL CLUB

Critically appraised topics (CATs) is a term regarding a document one creates for themselves in response to a clinical question. It summarizes an individual item of evidence one has found and presents the results in an easily readable format. General internal medicine fellows at McMaster University invented CATs as a means to sharpen their critical appraisal skills and improve their abilities as bedside teachers of evidence-based medicine. Several medical training programs maintain internet-based logs of papers that have met predetermined criteria for validity on various topics. Thus a database is created of current evidence for access by residents, fellows and faculty.

JOURNAL CLUBS AND ADULT LEARNING

Educational research has identified several unique characteristics of adult learners. These principles include: relate the task to personal goals or to the immediate environment, present learning objectives as clinical problems, use problem-solving strategies, vary teaching approaches to suit different learning styles, use active learner participation and provide frequent constructive feedback.¹⁶ It may be that instruction that incorporates these principles will enhance the learning experience for residents.²² Skills and behavior, not only knowledge, are likely to be improved when incorporating principles of adult learning.

CONCLUSIONS

Journal clubs are integral to graduate medical education, including most orthopedic training programs. This is true today as it was when journal clubs were first started. Successful journal clubs can be defined by many different criteria. Factors associated with high attendance and longevity of the club include mandatory attendance, availability of food, perceived educational value of the program director,²² having a committed leader, good faculty participation and attendance and formal instruction in epidemiologic principles.⁸,^36–38

Organizers of journal clubs should clearly state goals of the meetings and choose appropriate formats to achieve these goals. It is desirable to attain support from the departmental chairperson and other respected faculty.³⁸ Structured review instruments may increase levels of resident satisfaction with the educational experience and increase their understanding of the material and focus their presentations. By evaluating the journal club and responding to the needs of the participants, journal clubs can continue to evolve and improve to remain as a valuable part of the orthopedic residency training program.