Application of a Feedback Journal Club

The first documented journal club was organized in 1875 by William Osler at McGill University, to keep staff informed of medical science in a cost-effective manner.[1] Nowadays, journal club has already become a common form of interactive education in hope to keep medical practitioners up-to-date of current literature, understand research design and statistics, and expand critical appraisal skills. A considerable amount of literatures describing journal clubs in different forms have been published; most were to enhance students’ critical thinking, research competency, and evidence-based clinical practice. However, there is no gold standard to evaluate their effectiveness.[2] Some appraised the results according to the senior consultants’ assessment[3] while the others evaluate the journal clubs according to an attendee's self-report survey.[1,4] The form of journal club varies; and a well-accepted assessment system has yet been established.

We have modified the journal club into a feedback system to ensure maximum learning experience. Participants were encouraged to express their views on each article and open discussion among peers and supervisors were shaped. The estimated effects of the modified feedback journal club (FJC) were formulated based on the number of total publication and participant's survey.

Ever since August 2014, a weekly FJC has been hosted in the Department of Plastic Surgery, Peking Union Medical College Hospital. Each predetermined presenter selected a recent article 1-week prior. The selection of article had to be approved by a senior supervisor to ensure quality. One or two articles were discussed and at least two of the three senior supervisors must be present during each meeting.

The subjects of this research were young doctors in the Department of Plastic Surgery, including plastic surgical residents and doctoral/master degree students majoring in plastic surgery. Basic information such as the names of the presenter, attendees, discussion topic, and source of article were logged.

Two kinds of feedback modes were applied in the journal club. The first feedback form was received from peers and supervisors, followed by an open discussion. The presenters critically assessed the article according to the suggested peer-review guidelines[5,6] and shared deeper insights gained from the article for 15 min. The supervisors took charge of the open discussion; summarized all the diverse viewpoints and opinions based on their own research experience. At last, a consensus would be reached on whether/how to write a letter to the editor when suggestions were considered worthy.

Our journal club has included a segment called “experience from the first authors.” The first authors were invited for a face-to-face communication. The invited authors included: (1) those in the Department of Plastic Surgery who have published a research article; (2) Chinese authors who may not specialize in plastic surgery but recently published an article with a high impact factor; and (3) prestigious foreign professors specialized in plastic surgery. These authors were invited to share their experience in clinical or/and experimental research and presented their recent publication step-by-step.

The publications of participants as the first author were documented. Publications were divided into three categories: Chinese articles, Science Citation Index (SCI) articles, and SCI letters. The relationship between the times of participants’ attendance and the number of their publications was analyzed.

A confidential online survey for the quality evaluation of FJC was distributed to all participants in January 2017 through WeChat. The questionnaire consisted of 25 questions, covering participants’ basic information and Likert-scale questions.

A total of 113 times of FJC were held in 2.5 years, with a mean of 18.1 ± 6.4 participants attended each meeting. There were 29 participants who met our inclusion criteria as young doctors in the Department of Plastic Surgery. A total of 67 papers (29 in 2015 and 38 in 2016) were published, including 9 Chinese articles (2 in 2015 and 7 in 2016), 31 SCI letters (17 in 2015 and 14 in 2016), and 27 SCI articles (10 in 2015 and 17 in 2016). A positive correlation (R² = 0.71) was revealed between the times of FJC attendance and the number of publications (P < 0.01). All the 29 participants (16 males, 13 females) were asked to complete the survey within a 3-day period. Most of them were doctoral (14, 48.3%) and master (10, 34.5%) students of Peking Union Medical College. The rest (5, 17.2%) were residents. Participants at the age of 25–30 years made up the largest group (23, 79.3%), followed by those <25 years old (4, 13.8%), and those 31–35 years old (2, 6.9%). Not all of the participants were available for each attendance due to clinical work. Favorably, 21 participants (72.4%) attended more than half of the meetings. Among all, the 18 questions answered with Likert-scale scores, the most satisfied items were “the format of journal club organization” (4.44 ± 0.61), gains from the “experience from the first authors” section (4.41 ± 0.66), “peer-review checklist” provided for article analysis (4.38 ± 0.70), “skills of medical manuscript writing” (4.35 ± 0.60), and “enhanced critical thinking” (4.35 ± 0.65).

Journal club provides a forum to discuss new innovations and promotes improvement of clinical care, as well as to stimulate future research endeavors. In this research, a weekly journal club has been run for 3 years. The original format was like a closed-door lecture, which hampered its educational purpose. The holdup was similar to the previous studies.[7] The two most common obstacles faced were the limited experience shared and the lack of time. A novel approach was introduced aiming to gain perspectives and interactions. As most participants were students or junior surgeons, inadequate knowledge, and clinical experience demanded inputs from senior supervisors. For questions that could not be answered first hand, original authors of the discussed articles would be reached through a letter format or a face-to-face discussion. Twenty-six out of the 34 accepted letters were inquired, and the original authors responded with satisfactory answers in 21 out of 26 letters.

Medicine is rapidly advancing, finding a cost-effective method for teaching critical appraisal is challenging. Green and Johnson[8] were frontiers to teach critical appraisal and writing skills in a letter-to-editor manner. Participants were most pleased with this novel format; and we have had 34 letters published, which was a satisfactory result indicating progressive academic communication. However, further impact on medicine and research practice is still targeted. Participants without literature experience were advised to start from a letter first. Although only exemplary letters would be submitted for publication, skills were equally trained with the help of supervisors and peers. Numbers of participants have expressed that after several letter acceptance, they felt more confident and ambitious for future publications. At this time, they were encouraged to initiate their own research, ranging from systemic review, meta-analysis, experimental research, or clinical trial. Through the accumulation of knowledge and publication experience, the total publication within our journal club was 94.

Time is another barrier that should not be neglected.[7] As clinical practitioners, lack of time and time management are the key challenges. However, a platform for expressing ideas, communicating with peers and supervisors, initiating motivation, and generating benign competitions among peers are the reasons why they attend the FJC willingly despite heavy clinical work. What is inspiring is that most “senior” participants with more than 1-year experience in the journal club have achieved publications. Furthermore, the participants’ number of publications was positively correlated to the amount of attendance. Since there was no compulsory requirement on publication, all of the achievements were self-driven. Thus, publication could be a reliable and objective measurement for journal club evaluation.

In this research, the “journal club organization” earned the highest score, reflecting that this form of FJC was accepted. Not only did participants practice critical thinking and manuscript writing during the process, feedbacks from several sources had also benefited. The “experience from the first authors” was popular among participants, due to first-hand answers directly from the author. The lowest scored items were also acknowledged, and further improvements will be made in the future. Not everyone got to present every quarter; thus some participants have yet developed a regular reading habit. However, most of them reported that they would browse the topic, and read beyond the abstracts for the interesting topic. This was a good method of knowledge acquisition in a limited time. However, participants were not satisfied with statistical and English improvement. A multidisciplinary form, including the presence of biostatisticians and professional English medical editors, could be introduced later to better enhance discussion.

In conclusion, over the implementation of FJC, participants have received feedbacks from peers, supervisors and most importantly, original authors through multiple approaches. We have observed a significant improvement in both quantity and quality of participants’ publications. Anonymous survey has confirmed that the modified form of FJC was well accepted by the attendees, which was further proved to be an effective method to promote participants’ research ability with measurable outcomes.

Financial support and sponsorship

This work was supported by a grant from the Education Reforming Program, Peking Union Medical College (No. 2015zlgc0111).

Conflicts of interest

There are no conflicts of interest.