An Analysis of Successful Features of Anesthesiology Journal Clubs

Abstract

Background:

No studies have examined how journal clubs (JCs) are implemented in anesthesiology residency training programs. The goal of the study was to close this gap by (1) examining the format, content, and goals of JCs; (2) identifying features associated with higher resident attendance and JC success; and (3) examining program directors’ perspectives on JCs.

Methods:

A 41-question survey was sent to anesthesiology program directors. Answers were analyzed using multivariable logistic regression, multivariable linear regression, and exploratory factor analysis.

Results:

Out of 117 surveys sent across the United States, 80 program directors responded (68.4% response rate). Of the 80 programs, 77 (96.3%) programs have a JC, with 93.2% of them existing for more than 2 years. Most JCs (62.5%) neither formally appraised articles before meetings, nor formally evaluated their JC (59.7%). Faculty alone organized 44.4% and moderated 69.9% of the JCs. The role of residents was primarily limited to presenting selected articles with faculty guidance (83.3%). The average resident attendance was 49.7%. A multivariable linear regression analysis identified mandatory resident attendance, faculty turnout of >5 members, and longer intervals between JC meetings as features associated with higher resident attendance. Only 49.3% of JCs were successful as defined a priori by resident attendance >50% and longevity of ≥2 years. Features associated with JC success based on multivariable logistic regression included mandatory resident attendance and complimentary food.

Conclusions:

This largest survey of JCs in anesthesiology found that while JCs are widely established, half of them could be improved.

Introduction

The origin of a journal club (JC) can be traced to British surgeon Sir James Paget, describing a meeting at St Bartholomew’s Hospital in London during 1835–1854 as “a kind of club, in a small room over a Baker’s shop near the Hospital-gate where we could sit and read the journal.”¹ The first formal JC was established in 1875 by Sir William Osler at McGill University “for the purchase and distribution of periodicals to which he could ill afford to subscribe.”^2,3

The purpose of a JC has transformed over time. In 1999, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project required teaching and assessing 6 core competencies. Many programs turned to JCs to incorporate them.^3–6 The evolution of evidence-based medicine further altered many JCs.⁷ The role of JCs has expanded to include imparting knowledge, teaching critical appraisal, developing clinical decisionmaking, promoting lifelong learning, enhancing communication skills, reviewing the literature, and promoting social interactions.^2,6

Studies have identified successful characteristics and provided recommendations on how to effectively implement JCs,^7–14 but none have been conducted in anesthesiology. While articles describe techniques for JCs in anesthesiology to be implemented, improved,¹⁵ used as a teaching tool,^4,6 or even Twitter-augmented,¹⁶ none have identified characteristics of successful anesthesiology JCs or provided recommendations on how to implement or optimize them.

To close this gap, this study surveyed 117 anesthesiology program directors (PD) to provide information on anesthesiology JCs from the long-term PD educator’s perspective. Trainees were not surveyed as part of this study.

The goals of the study were to (1) examine the format, content, goals, and organization of JCs in anesthesiology residency training programs (ARTPs) across the United States; (2) identify key features associated with higher resident attendance and JC success; (3) examine PDs’ perspectives on problems facing JCs and strategies to improve JCs; and (4) explore the role of JCs in teaching ACGME core competencies.

Materials and Methods

Survey Creation and Distribution

As there are no established surveys to evaluate JCs, a questionnaire was developed over several months by a group of 8 to 10 ARTP directors, experienced teaching faculty, and members of the Resident Education Committee of the Society for Education in Anesthesia. The dominant contributors became the authors.

Survey creation was initiated with an in-person meeting:

• (1) The planned goal of the survey was to determine the purpose, goals, frequency, location, content, design, evaluation, relevance, and improvement strategies of JCs in ARTPs across the United States.

• (2) The consensus was to focus on anesthesia PDs and their experienced, long-term educator’s perspective. It was believed that the cyclical, short-term perspective of learners may represent general satisfaction and individual preference, not educational success. There was also concern for response bias if only a subset (satisfied or unsatisfied) of residents responded.

• (3) The literature review revealed a well-established definition of JC success based on resident attendance of more than 50% and a continuous existence of at least 2 years.⁹ Other definitions of JC success^12,17 were considered but ultimately were dropped for 3 main reasons: (a) currently, no established methods exist that reliably quantify the impact of educational interventions on the long-term skills or patient care; (b) resident satisfaction has not been found to be related to quality of education; and (c) most ARTPs do not collect such data; consequently, it would reduce the survey response rate and the generalizability of results.

• (4) Questions were developed in 6 rounds of discussions; consensus was obtained by two-thirds majority. The survey consists of 3 sections: residency program background, JC format, and content/purpose. The questionnaire was reviewed by the group and then piloted. The final survey contained 41 questions, including several for PDs to comment on their strategies, current problems, and new implementation (Appendix A).

An anonymous survey was distributed via SurveyMonkey (Palo Alto, CA) to anesthesia PDs. A total of 117 out of 131 ACGME-approved ARTPs in the United States were surveyed, as only programs with publicly available email addresses who had not previously opted out of SurveyMonkey were contacted. Follow-up reminders were sent to nonresponders at 6 weeks and again at 10 weeks.

Statistical Analysis

Frequencies and percentages of responses to each survey question were calculated and presented to summarize the program and JC-related features. To identify features associated with higher average resident attendance and JC success, survey question responses relating to the format of JCs and program characteristics were selected and coded as categorical variables (as predictor variables) for multivariable linear regression and logistic regression analyses (Appendix B).

Average resident attendance (%) at a JC was calculated for individual programs using the formula below. The average number of residents attending a JC is the mean of the answer range to the survey question number 18 (Appendix A; see equation).

JC success was defined in this study as average resident attendance of more than 50% and a continuous existence for at least 2 years. This binary outcome was our primary response variable. The average resident attendance (%) was our secondary response variable.

To characterize features associated with JC success, χ² or Fisher exact tests between successful and unsuccessful JCs were conducted. Exploratory factor analysis was performed to characterize underlying factors in our survey. Model selections in multivariable logistic regression and in multivariable linear regression were performed to identify features associated with JC success and higher average resident attendance (%), respectively (explained in detail below).

Exploratory Factor Analysis

Since survey responses were coded as categorical data, the correlation matrix was generated and used as the input data in exploratory factor analysis. Principal factor analysis was applied to identify latent factors. The number of factors was determined by scree plot and criteria that eigen values are larger than 1.

Multivariable Logistic Regression and Linear Regression on Imputed Datasets

Multiple-choice features were transformed into binary variables to improve regression model efficiency. The reference category was the most prevalent response, or the most normative group as defined in current literature. There were 14 variables with 1 to 3 missing values. Five multiple imputations were performed to address missing values using the fully conditional specification method.¹⁸ Stepwise selection was then used to select variables from all independent variables of interest and their two-way interactions based on the Schwarz-Bayesian information criterion.

To minimize the separation and overfitting issues in the logistic regression model, a penalized likelihood method was applied to the final model. The overall fit of the final model was assessed using the Hosmer-Lemeshow test and the receiver operating characteristic curve. Odds ratios (ORs) and 95% confidence intervals (CI) were reported for selected variables.

The model fit of the multivariable linear regression model was assessed with the adjusted coefficient of determination (R²). Fisher r to z transformation was used to calculate multiple imputation estimates of the adjusted R² as suggested by Harrell.¹⁹ Collinearity of independent variables was assessed by examining Variance Inflation Factor. Residual plots were evaluated to verify the assumption of independence, constance, and normality in the error terms. The influence of individual observations was considered using Cook distance. Finally, coefficient estimates and their standard errors are presented for selected variables.

All results are uncorrected for multiple comparisons. P values <.05 were considered statistically significant, and all statistical analyses were performed using SAS version 9.4 (SAS Institute, Inc, Cary, NC).This article adheres to the applicable Equator guidelines.

Results

Out of 117 surveys sent to directors of ACGME-approved US ARTPs, 80 PDs responded (68.4% response rate). Of responders, 77 (96.3%) had active in-person JCs incorporated into ARTP: 60 programs at a university hospital (77.9%), 12 programs at a university-affiliated hospital (15.6%), and 5 at a freestanding community hospital (6.5%). Seventy-three programs provided information on their JC’s longevity, with 68 (93.2%) existing for more than 2 years. Three programs no longer offering a JC (3.8%) cited “not enough personnel” or “implementation of ACGME competencies by different methods.”

JC Format, Content, Purpose, and Relevance

Reported JCs commonly met on campus only (67.1%), once a month (54.9%), and in the evening (63.9%; Appendix B). Only 30.6% met in the morning and 5.6% at midday. The majority (57.5%) met for 1 hour, 37.0% for 2 hours, and 5.5% for 3 hours. Complimentary food was offered in 45 of the 70 JCs that answered (64.3%). The 3 most important goals of the JC reported by the PDs were “critical appraisal”, “research design and methodology”, and “clinical innovations” (97.2%, 93.0%, and 91.5%, respectively).

Most JCs neither formally appraised articles before meetings (62.5%), nor formally evaluated their JC (59.7%). About half (47.9%) set annual goals. Almost all JCs (98.6%) chose articles right before the JC session (Appendix B).

Residents were rarely given full responsibility for organizing the JC (1.4%), moderating sessions (1.4%), choosing articles (1.4%), or presenting articles (11.1%). Faculty alone selected articles in 50%, organized 44.4%, and moderated 69.9% of the JCs. The role of residents was limited to presenting articles with faculty guidance (83.3%) (Appendix B).

JC was considered to be important (50.7%) or very important (32.4%) by 83.1% of all PDs. Compared to those who ranked JCs as somewhat or not important, they were more likely to incorporate an active learner model (90% vs 64% respectively, P = .04) and provide content focused on research design (97% vs 75% respectively, P = 2.02). They were also more likely to believe residents viewed JCs as important or very important (51% vs 0% respectively, P = .001). PDs who believed residents view JCs as important or very important were more likely to incorporate goal-setting compared to those who believed residents perceive JCs as somewhat or not important (63% vs 37% respectively, P = .03).

Of the 72 programs reporting information on faculty attendance, 35 programs (48.6%) had <5 faculty members attend JC meetings. Compared to JCs with ≥5 faculty attendance, these JCs were shorter (1 hour vs ≥2 hours, P = .008), presented fewer articles per session (<3 vs ≥3 articles, P = .026), and met on campus (vs off-campus, P = .035) without complimentary food (vs with food, P = .031).

Exploratory Factor Analysis

To understand the underlying factors of survey answers and explain the variances, exploratory factor analysis was applied to our variables (Appendix B). It identified 4 latent factors that explained 83% of variance in the data: (1) JC format; (2) importance and educational structure; (3) preparation and resident attendance; and (4) involvement of residents (31.5%, 21.5%, 16.0%, and 13.7%, respectively; Figure 1).

Figure 1.

The 4 latent factors identified by exploratory factor analysis. The exploratory factor analysis identified 4 latent factors that explained 83% of variance in the data. The 4 factors are outlined with percentage of variance explained by each factor in each oval on the left. The variables and their loadings are listed for each factor on the right: Factor 1 described the format of journal club (JC; 31.5% of data variance): Longer duration with higher number of discussed articles, evening time of the day, complimentary food, and higher faculty attendance were positively correlated with Factor 1. On-campus only location had negative correlation. Factor 2 described the importance and educational structure of JCs (21.5% of data variance): Higher importance rating of JCs, presented by faculty and more structure had positive correlation with Factor 2. Factor 3 included the preparation for and attendance of the JC (16.0% of data variance): Formal appraisal of articles, as well as mandatory and recorded attendance had positive association with Factor 3. Factor 4 characterized resident involvement in JC (13.7% of variance): Increased resident participation in selecting articles and JC moderation had positive correlation with Factor 4. Higher faculty attendance had positive correlation, while higher frequency of JCs a negative correlation with Factor 4. The result is based on the correlation matrix of variables, used principal factor analysis method to extract factors with priors = squared multiple correlation, the number of factors were determined by scree plot and eigen values >1.

^a Loadings for Factor 1, Factor 2, and Factor 4.

^b Loadings for Factor 2 and Factor 3.

Features Associated With Higher Resident Attendance

The average resident attendance was 49.7% (median 52.7%), despite resident attendance being mandatory in 69.4% and recorded in 88.9% of the JCs. The range of reported resident attendance was broad, with 13.9% reporting an average attendance of ≤25%; 34.7% reporting average attendance between 25.1% and 50%; 43.1% reporting average attendance between 50.1% and 75%; and 8.3% reporting average attendance of more than 75% (Appendix B).

To identify JC features associated with higher average resident attendance, univariable and multivariable linear regressions were performed using the features listed in Appendix B, adjusting for the size of the ARTP (Table 1). The univariable analysis found 6 features associated with higher resident attendance (Table 1), with 3 remaining significant using multivariable analysis (R² = 0.75): meeting monthly (P = .005) or less frequently (P = .001), mandatory resident attendance (P = .007), and faculty attendance of 5 or more members (5–10 members: P = .033; 11–15 members: P = .014). These features are parts of Factor 1, Factor 3, and Factor 4 based on exploratory factor analysis. To explore the relationship between higher average resident attendance and faculty attendance further, a post-hoc test was conducted, demonstrating a positive linear trend between increasing average faculty attendance and higher average resident attendance (P = .036; Figure 2).

Table 1.

Journal Club (JC) Features Associated With Higher Average Resident Attendance

Variable Name	Univariable Analysis			Multivariable Analysisa
	Parameter Estimate (SE)	t Value	P Valueb	Parameter Estimate (SE)	t Value	P Valueb
Location: on campus onlyc	−11.2 (5.6)	−2.01	.045
Frequencyd
Monthly	26.6 (7.5)	3.55	<.001	12.2 (4.3)	2.80	.005
Less frequently than monthlye	27.9 (7.9)	3.51	<.001	15.3 (4.7)	3.26	.001
Time of day: eveningf	6.2 (5.5)	1.12	.264
Durationg
2 h	9.4 (5.2)	1.82	.068
>2 h	35.2 (10.8)	3.25	.001
Complimentary food available	1.7 (5.9)	0.29	.773
Formal method of appraising articles	4.1 (5.4)	0.76	.444
Formal JC evaluation method	1.7 (5.4)	0.32	.752
Goals set at beginning of year	1.3 (5.3)	0.25	.800
Organizer: residents or residents with facultyh	7.1 (5.3)	1.34	.181
Moderator: residents or residents with facultyh	0.6 (5.7)	0.10	.922
Chooser of articles: residents or residents with facultyh	5.2 (5.3)	0.98	.326
Presenter of articles: residents or residents with facultyh	−4.6 (11.0)	−0.42	.676
No. of articles presentedi
3–4	9.4 (5.2)	1.82	.069
5–6	42.5 (15.1)	2.82	.005
Mandatory resident attendance	13.9 (5.7)	2.44	.015	8.4 (3.1)	2.69	.007
Recorded resident attendance	−5.6 (8.3)	−0.67	.503
Faculty attendancej
5–10	12.5 (5.3)	2.35	.019	6.4 (3.0)	2.14	.033
11–15	24.0 (8.2)	2.93	.005	12.1 (4.9)	2.47	.014
Importance rating of JC as part of the residency training by anesthesia residency program directork
Very important	−42.1(22.2)	−1.89	.059
Important	−36.1 (22.1)	−1.64	.102
Somewhat important	−33.3 (22.7)	−1.46	.143
Importance rating of JC as part of the residency training by the residents as perceived by the anesthesia residency program directork
Very important	−4.7 (14.4)	−0.33	.743
Important	−2.5 (10.2)	−0.24	.810
Somewhat important	−1.4 (9.9)	−0.14	.886

^a Final model after forward-backward stepwise selection with Schwartz-Bayesian information criterion, with adjustment for program size.

^b Boldface values are signnificant.

^c Reference: off-campus only or both on-campus and off-campus (program directors could choose on-campus only, off-campus only, or both).

^d Reference: multiple times a month.

^e May include quarterly, every other month, or 6–10 times a year.

^f Reference: morning + midday.

^g Reference: 1 hour.

^h Reference: faculty only.

ⁱ Reference: 1–2 articles.

^j Reference: <5.

^k Reference: not important.

Figure 2.

Mean resident attendance (%) vs average faculty attendance (N). A post-hoc test for linear trend on faculty attendance was conducted, controlling for program size, mandatory attendance, and journal club (JC) frequency as reported by program directors. A positive linear trend was observed between increasing average faculty attendance and average resident attendance (P = .036), with model-implied least square means of percent resident attendance (SE) of 45.07% (1.95%), 50.56% (2.27%), and 56.35 % (4.30%) for <5, 5–10, and 11–15 faculty, respectively. Error bars show standard error of the mean.

When asked about average faculty attendance, program directors were given 3 choices: <5, 5–10, or 11–15; mean values (3, 8, and 13, respectively) were used for mean resident attendance calculations.

Features Associated With JC Success

Overall, 73 programs provided information on JC resident attendance and longevity. Thirty-six (49.3%) met the definition of a successful JC: JC resident attendance more than 50% and a continuous existence for at least 2 years.

Successful JCs (Table 2) demonstrated significantly higher resident attendance (67.8 ± 12.4% vs 31.6 ± 11.8%, respectively; P < .001) and mandatory resident attendance (85.7% vs 54.1%, respectively; P = .004).

Table 2.

Features Associated With Journal Club (JC) Success ^a

Features and Choices	Programs With Data on Resident Attendance and Longevityb (n = 73)c	JCs With Resident Attendance >50% and Continuous Existence ≥2 Years (n = 36)	JCs With Resident Attendance ≤50% and/or Continuous Existence <2 Years (n = 37)	P Valued
Location				.038
On-campus onlye	49 (67.1)	20 (55.6)	29 (78.4)
Off-campus only or both on-campus and off-campuse	24 (32.9)	16 (44.5)	8 (21.6)
Frequency				.002
Multiple times a month	9 (12.7)	0 (0.0)	9 (25.0)
Monthly or less frequently	62 (87.3)	35 (100.0)	27 (75.0)
Time of day				.074
Morning + midday	26 (36.1)	9 (25.7)	17 (46.0)
Evening	46 (63.9)	26 (74.3)	20 (54.0)
Duration				.007
1 h	42 (57.5)	15 (41.7)	27 (73.0)
>1 h	31 (42.5)	21 (58.3)	10 (27.0)
Complimentary food available				.400
Yes	45 (64.3)	24 (68.6)	21 (60.0)
No	25 (35.7)	11 (31.4)	14 (40.0)
Formal method of appraising articles				.224
Yes	27 (37.5)	16 (44.4)	11 (30.6)
No	45 (62.5)	20 (55.6)	25 (69.4)
Formal JC evaluation method				.471
Yes	29 (40.3)	13 (36.1)	16 (44.4)
No	43 (59.7)	23 (63.9)	20 (55.6)
Goals set at beginning of year				.556
Yes	34 (47.9)	18 (51.4)	16 (44.4)
No	37 (52.1)	17 (48.6)	20 (55.6)
Organizer				.225
Faculty only	32 (44.4)	13 (37.1)	19 (51.4)
Residents alone or with faculty	40 (55.6)	22 (62.9)	18 (48.7)
Moderator				.939
Faculty only	51 (69.9)	25 (69.4)	26 (70.3)
Residents alone or with faculty	22 (30.1)	11 (30.6)	11 (29.7)
Chooser of articles				.346
Faculty only	36 (50.0)	16 (44.4)	20 (55.6)
Residents alone or with faculty	36 (50.0)	20 (55.6)	16 (44.4)
Presenter of articles				.331
Faculty only	4 (5.6)	1 (2.9)	3 (8.1)
Residents alone or with faculty	68 (94.4)	34 (97.1)	34 (91.9)
No. of articles presented				.006
1–2	44 (61.1)	16 (45.7)	28 (75.7)
>2	29 (39.7)	20 (55.6)	9 (24.3)
Average resident attendance, mean ± SD, %	49.7 ± 21.8	67.8 ± 12.4	31.6 ± 11.8	<.0001
Mandatory resident attendance				.004
Yes	50 (69.4)	30 (85.7)	20 (54.1)
No	22 (30.6)	5 (14.3)	17 (46.0)
Recorded resident attendance				.473
Yes	64 (88.9)	30 (85.7)	34 (91.9)
No	8 (11.1)	5 (14.3)	3 (8.1)
Faculty attendance				.058
<5	35 (48.6)	13 (37.1)	22 (59.5)
5 or more	37 (51.4)	22 (62.9)	15 (40.5)
Importance rating of JC by program director				.187
Very important or important	59 (83.1)	27 (77.1)	32 (88.9)
Somewhat or not important	12 (16.9)	8 (22.9)	4 (11.1)
Importance rating of JC by residents as perceived by program director				.633
Very important or important	30 (41.7)	14 (38.9)	16 (44.4)
Somewhat important or not important	42 (58.3)	22 (61.1)	20 (55.6)

^a Of the 77 programs conducting journal club regularly, 73 provided information on resident attendance and longevity. A successful JC was defined based on resident attendance of more than 50% and a continuous existence of at least 2 years. Data are presented as number (%) unless otherwise indicated.

^b Resident attendance was defined as the ratio between the average number of anesthesia residents attending a JC and the total number of residents enrolled in the program.

^c Based on the numbers of programs reporting longevity and resident attendance. Value of n varied because of the different response rate to different questions.

^d χ² test or Fisher exact test. Boldface values are signnificant.

^e Program directors could choose between on-campus only, off-campus only, or both.

Nonsuccessful JCs were shorter, with 73.0% meeting for only 1 hour (successful JCs: 41.7%; P = .007), 75.7% presented only 1 to 2 articles (successful JCs: 45.7%; P = .006), and met on-campus (78.4% vs 55.6%; P = .038). None of the JCs that met multiple times a month met the criteria of a successful JC (P = .002).

Univariable and multivariable logistic regression analyses were performed to identify features associated with JC success, adjusting for the size of the ARTP (Table 3). The univariable analysis identified 6 features associated with JC success (Table 3), but only 2 remained significant using multivariable analysis (area under curve = 0.935): mandatory resident attendance (OR 6.9 [CI: 1.36–34.7]; P = .020) and complimentary food (OR 5.5 [CI:1.06–28.3]; P = .042), which belong to Factor 3 and Factor 1 based on the exploratory factor analysis, respectively.

Table 3.

Journal Club (JC) Features Associated With Success ^a

Variableb	Univariable Analysis		Multivariable Analysisc
	OR (95% CI)	P Valued	OR (95% CI)	P Valued
Location: on-campus onlye	0.35 (0.12–0.96)	.041
Frequency: monthly or less frequentlyf	27.9 (1.36–570.3)	.031
Time of day: eveningg	2.2 (0.83–5.9)	.111
Duration: >1 hh	3.8 (1.41–10.1)	.008
Complimentary food available	1.48 (0.55–4.0)	.441	5.5 (1.06–28.3)	.042
Formal method of appraising articles	1.80 (0.68–4.7)	.236
Formal JC evaluation method	0.71 (0.27–1.83)	.479
Goals set at beginning of year	1.34 (0.53–3.4)	.535
Organizer: residents or residents with facultyi	1.70 (0.66–4.3)	.268
Moderator: residents or residents with facultyi	1.04 (0.38–2.8)	.939
Chooser of articles: residents or residents with facultyi	1.54 (0.60–3.90)	.367
Presenter of articles: residents or residents with facultyi	2.0 (0.22–18.4)	.540
No. of articles presented: >2j	3.9 (1.43–10.6)	.008
Mandatory resident attendance	4.6 (1.48–14.5)	.009	6.9 (1.36–34.7)	.020
Recorded resident attendance	0.52 (0.11–2.4)	.404
Faculty attendance: >5k	2.6 (1.01–6.7)	.048
Importance rating of JC as part of the residency training by anesthesia residency program directorl	0.43 (0.11–1.65)			.219
Importance rating of JC as part of the residency training by the residents as perceived by the anesthesia residency program directorl	0.80 (0.31–2.1)			.642

Abbreviations: CI, confidence interval; OR, odds ratio.

^a A successful journal club was defined based on resident attendance of more than 50% and a continuous existence of at least 2 years.

^b Design levels for multiple choice features were transformed into binary variables to improve model efficiency. The most prevalent choice or most normative group was taken as the reference category.

^c Final model after stepwise selection, adjusting for program size.

^d Boldface values are signnificant.

^e Reference: off-campus (included off-campus only and off-campus and on-campus; program directors had 3 choices: on-campus only, off-campus only, or both).

^f Reference: multiple times per month.

^g Reference: morning + midday.

^h Reference: 1 hour.

ⁱ Reference: faculty only.

^j Reference: 1–2 articles.

^k Reference: <5.

^l Reference: somewhat or not important.

PDs’ View

Seventy-two PDs provided opinions on characteristics of successful JCs. The top 4 characteristics included: (1) faculty facilitator (81%); (2) discussion of clinical articles (67%); (3) complimentary food (46%); and (4) large number of participants (33%). In the comment section, most PDs emphasized that active involvement of trainees alongside motivated senior faculty guiding a “candid discussion” of “good articles” makes their JC successful.

Strategies to Improve JCs

The survey provided opportunities for PDs to include free-text responses such as novel approaches or recent changes. Of the 39 PDs who responded, 12 (30.8%) introduced steps to enhance quality by including goals and objectives, critical appraisal tools, structured evaluation, or inviting experts; 4 improved web access to articles and/or the JC; 2 changed the time or duration of the JC; and 1 included core competencies of professionalism and system-based practice.

PDs commented they would like to improve their JCs by increasing resident and faculty attendance, adding tools for critical appraisal, and selecting “better articles.” However, half of the responding PDs (51%) did not plan to modify their JCs.

JC as a Tool to Teach ACGME Core Competencies

The PDs were asked to rank the 6 ACGME core competencies according to their emphasis in their JCs on a scale between 1 (least emphasized) and 6 (most emphasized). Practice-based learning and improvement was ranked 6 (the highest) by 39.7% of the PDs, followed by medical knowledge and patient care (25.8% and 23.1%, respectively). Most programs (71.7%) ranked interpersonal and communication skills between 2 and 4. Professionalism and systems-based practice were ranked the least (1 or 2, 55.7% and 49.3%, respectively).

Discussion

This study, the largest survey of JCs in ARTPs, found 96% of responding programs regularly conduct a JC, with 93.2% of them established for more than 2 years. This is consistent with other specialties (range: 84%–100%).^{1,8–10,13,20} Despite its omnipresence and perceived importance as an integral part of residency training, only 49% of anesthesiology JCs were successful, with an average resident attendance of 49.7% that was lower than in other specialties.^1,8,9

Multivariable logistic regression analysis adjusted for ARTP size identified mandatory resident attendance and complimentary food as characteristics of successful JCs. Features associated with higher resident attendance included: mandatory resident attendance, faculty turnout of ≥5, and monthly (or less) JC meetings. Other features relevant for JC success or higher resident attendance based on univariable analyses included longer sessions (>1 hour), presenting ≥3 articles, and off-campus location. Finally, exploratory factor analysis identified 4 latent factors in the survey data: format, importance and educational structure, resident attendance, and involvement of residents in JC.

The results of the current study suggest that half of JCs could be improved. PDs confirmed awareness of low resident and faculty attendance, yet only half (49%) planned changes. Our results offer some simple but much-needed recommendations to establish or improve an existing JC (summarized in Table 4).

Table 4.

Recommendations to Improve an Existing or Start a New Journal Club (JC) ^a

Resident attendance should be mandatory and recorded

Faculty attendance should be encouraged by making it easy for everyone to participate

• Scheduling consistent JC meetings (eg, every first Tuesday of the month)

• Providing location with easy parking and accommodating different food preferences

Educational structure should be clearly defined, including

• JC goals, (eg, Focus on epidemiology or Professionalism in the OR)

• Timely dissemination of articles with formal method(s) to critically appraise them

• Formal evaluation at the conclusion of every JC

Active learner participation should be encouraged

• Faculty mentors should act in a background-supportive role

• Residents should be encouraged to moderate or comoderate the journal club

The dedicated faculty JC facilitator should focus on the administrative duties

• Develop JC structure and delineate formal methods to critically appraise articles

• Be the go-to person with any issues related to the JC (eg, selection of experts)

• Set up the location, ensure distribution of articles, and arrange food and parking

• Overbearing JC faculty facilitator involvement should be avoided

Complimentary food should be provided (accommodating different food preferences)

JC format should be chosen based on the goal of the JC

• Longer sessions in the evening allow a longer, more interesting, and educational discussion in a relaxed and social atmosphere, facilitating team bonding

• Shorter sessions in the morning lack the time for meaningful discussions and have worse attendance because of time conflicts with scheduled surgeries. Such sessions should focus on concise knowledge transmission (less discussion—more key facts)

^a A high-perceived educational value by the participants is essential for JCs to be successful. Literature and our results suggest that active participation by residents, involvement of experts in epidemiology and statistics, clear educational structure, longer duration (>1 hour) allowing “candid” discussion of “good” and clinically relevant articles, as well as social-relaxed atmosphere increase the educational value perceived by participants.^{2,7–10,12,14,21–24}

Attendance, Roles, and Contributions

The importance of mandatory resident attendance for JC success in this study is consistent with other studies.^{7,9,11,13–15,21,22} Notably, nearly half of all anesthesia JCs suffered from low (<5) faculty participation. Strong faculty attendance is an important factor of JC success across specialties,^{9,10,20,22,25} by enhancing resident attendance, quality of discussions, and role modeling.^{9,10,20,25,26}

Anesthesia residents had limited involvement in the organizational process of the JC, primarily presenting articles under faculty guidance. This finding might reflect the belief among PDs that a strong faculty facilitator is essential for JC success. Nonetheless, our data seem to suggest that a strong faculty facilitator without active resident involvement might be counterproductive to JC success. There is growing evidence that encouraging active learning by permitting resident independence while supported by committed faculty, is crucial for the success of the JC.^{10,23,26–28} Active inclusion of residents increases resident attendance,¹² reading time,²⁸ and likelihood of reading the selected articles,^21,26 as well as residents’ ability to apply the learned methodology to patients.²⁷

Educational Structure

Surprisingly, though consistent with findings reported in other specialties, there was a lack of clearly defined educational structure in most anesthesia JCs despite the belief that it is essential to JC success.^5,15,26 Formal evaluation and assessment increase resident satisfaction and improve the perceived educational value of a JC without increasing residents’ workload.²⁴ Structured instruments for review of articles improve critical appraisal skills, resident satisfaction, and perceived educational value of the JC without increasing resident workload or decreasing attendance.^6,24,29

Format

Features associated with higher attendance or JC success identified in our study may be explained by social learning theory,³⁰ especially as anesthesiology allows few opportunities for resident and faculty interaction. Our results, emphasizing the importance of high attendance and complimentary food for JC success, highlight this social aspect of the JC fostering bonding⁸ and are consistent with other studies.^{7,9,11,13–15,21,22}

Off-campus JCs with food, longer duration, and more active resident involvement, may allow for a more comprehensive discussion of multiple articles with faculty and a less stressful environment with a strong social component.^8,9 Ultimately, such format provides a powerful blend of education and role-model learning in conjunction with social bonding. Social bonding has been found to predict JC success.^7–9,14 Furthermore, limiting JC’s frequency may maintain a high level of motivation while preventing a loss of interest.^10,15

In contrast, morning JCs were more faculty-run (organization and presentation of articles). shorter, and met more frequently. The lower resident attendance was likely because of time conflicts with scheduled surgeries, thereby also decreasing time for meaningful discussion.

ACGME Core Competencies

Since the introduction of the ACGME core competencies in 1999, many residency programs turned to JC as a facilitative tool.^3–6 Shakespeare et al⁴ described using the anesthesia JC to teach professionalism. The current study found that the overwhelming majority of anesthesia JCs focus on practice-based learning and improvement, medical knowledge, and patient care. Interpersonal and communication skills, professionalism, and systems-based practice were only moderately included in JC sessions, suggesting that the JC remains an untapped opportunity to teach all 6 ACGME core competencies and their related milestones.

Limitations

There are several limitations to this study. First, only programs with publicly available email addresses who had not previously opted out of SurveyMonkey were contacted. Second, the intentional brevity of the survey limited the ability to cover all areas of interest in depth. Third, selection bias cannot be excluded. Fourth, the survey was distributed to anesthesiology residency PDs only, as we were primarily interested in the experienced, long-term educator’s perspective. It is possible that trainee perspective on JCs may have contributed additional valuable insights. Fifth, this study used a well-established definition of JC success based on easily obtainable data on resident attendance and longevity,⁹ but attendance was self-reported by PDs, which may be prone to error, especially as resident attendance was reported by multiples of 5 rather than exact numbers. Besides, the denominator may have been influenced by residents on leave, vacation, or off-site rotations. While it is possible that other definitions of JC success, like resident satisfaction or potential markers of quality of education, may have provided additional insights, such defintions have their own limitations. For example, resident satisfaction has not been found to be related to quality of education, and currently no established methods exist that reliably quantify the impact of a particular educational intervention on the long-term skills or patient care. Finally, the survey was designed to ensure anonymous participation; therefore, the characteristics or demographics of the programs responding to the survey were not collected, which may impact the generalizability of the results.

Conclusions

Nearly every ARTP that responded to the survey had a JC with regular meetings, but only about half of the programs met the qualifications of a successful JC. Most anesthesiology JCs were organized solely by faculty members and lacked distinct educational structures. Mandatory resident attendance and complimentary food were associated with JC success. Features associated with higher resident JC attendance included mandatory resident attendance, higher faculty turnout, and monthly or less meeting frequency intervals.

Appendices

Appendix A. Questionnaire

• I. Residency Program Background

  • 1. What kind of setting is your residency program in?

    • □ University hospital

    • □ University affiliated community hospital

    • □ Freestanding community hospital

    • □ Other (please specify):___________________________________________________

  • 2. How many anesthesiology residents does your program train annually?

    • □ CA-1

    • □ CA-2

    • □ CA-3

  • 3. Do you regularly conduct journal clubs to enhance your anesthesiology residency program?

    • □ Yes

    • □ No

      If your answer is “Yes”, please go to Section II.

  • 4. When did your journal club end?

    ________________________________________________________________________

  • 5. Why did your journal club end? (Select all that apply)

    • □ Not enough personnel

    • □ Not enough attendance

    • □ Implementation of ACGME core competencies required different teaching methods

    • □ Other reason (please specify): _____________________________________________

  • 6. Have you tried to restart your journal club?

    • □ Yes

    • □ No

  • 7. Why do you think did your journal club not succeed?

    ________________________________________________________________________

• II. Organization of Journal Club

  • 8. How long has your journal club been in existence?

    • □ <6 mo

    • □ 1 y

    • □ 2 y

    • □ 3 y

    • □ >4 y

  • 9. How often does the journal club meet?

    ________________________________________________________________________

  • 10. What is the average length of time for a journal club meeting (h)?

    • □ 1

    • □ 2

    • □ 3

    • □ >3

  • 11. What time of the day does the journal club meet?

    • □ Morning

    • □ Midday

    • □ Evening

  • 12. Where does the journal club meet? (Select all that apply)

    • □ On campus/in the hospital

    • □ Personal home

    • □ Restaurant

    • □ Other (please specify): ___________________________________________________

  • 13. Who organizes your journal club?

    • □ Residents

    • □ Faculty

    • □ Residents and faculty

  • 14. Who is the moderator of the journal club meeting?

    • □ Faculty member

    • □ Resident

    • □ Faculty member and resident

  • 15. Who chooses the articles that are to be presented? (Select all that apply)

    • □ Residents

    • □ Faculty

  • 16. Who presents the articles?

    • □ Junior residents

    • □ Chief residents

    • □ Faculty

    • □ All residents with faculty guidance

    • □ Other (please specify): ___________________________________________________

  • 17. Which residents participate in your journal clubs? (Select all that apply)

    • □ CA-1

    • □ CA-2

    • □ CA-3

  • 18. How many residents attend the journal club meetings on average?

    • □ <5

    • □ 5–10

    • □ 11–15

    • □ 16–20

    • □ 21–25

    • □ >26

  • 19. How many articles does the journal club discuss every meeting?

    • □ 1–2

    • □ 3–4

    • □ 5–6

  • 20. Who receives copies of the journal club articles for review before the meeting?

    • □ Everyone

    • □ All the residents

    • □ The presenters

    • □ Faculty

  • 21. Is the residents’ attendance mandatory?

    • □ Yes

    • □ No

  • 22. Is attendance of all participants recorded at your journal club?

    • □ Yes

    • □ No

  • 23. How many faculty members attend the journal club meetings on average?

    • □ <5

    • □ 5–10

    • □ 11–15

  • 24. Besides residents and attendings, who else attends the journal club? (Select all that apply)

    • □ Medical students

    • □ Fellows

    • □ Invited guest speakers

    • □ CRNAs

    • □ Other (please specify):___________________________________________________

  • 25. Is complimentary food provided at your journal club?

    • □ Yes

    • □ No

• III. Content and Purpose of Journal Club

  • 26. How important do you rate journal clubs as a part of your residency training?

    • □ Not important

    • □ Somewhat important

    • □ Important

    • □ Very important

  • 27. How important do you think the residents rate journal clubs as a part of their training?

    • □ Not important

    • □ Somewhat important

    • □ Important

    • □ Very important

  • 28. How do the participants of the journal club appraise the articles before the meetings?

    • □ Reading guide/checklist

    • □ No formal method

    • □ Other (please specify): __________________________________________________

  • 29. Do you set the goals of your journal club at the beginning of the year?

    • □ Yes

    • □ No

  • 30. When are the articles chosen for the journal club?

    • □ At the beginning of the year

    • □ The week/month/quarter before the next one

  • 31. What is the method of evaluation of the journal club?

    • □ No formal evaluation

    • □ Knowledge assessment

    • □ Skills assessment

    • □ Attitude assessment

  • 32. Which content is covered in your journal club? (Select all that apply)

    • □ Critical appraisal

    • □ Epidemiology and statistics

    • □ Research design and methodology education

    • □ Clinical innovations, procedures and treatment

  • 33. Which teaching methods are employed in the journal club? (Select all that apply)

    • □ Structured session (allotted time for presentation and for critical discussion)

    • □ Checklist and reading guides

    • □ Facilitated discussion by resident

    • □ Facilitated discussion by attending

    • □ Other (please specify): ___________________________________________________

  • 34. Which of these adult learning principles do you make use of in your journal club? (Select all that apply)

    • □ Presentation of learning objects in patient cases/actual situations

    • □ Problem solving

    • □ Multiple teaching formats

    • □ Active learner involvement

    • □ Use of frequent constructive feedback

  • 35. Are journal clubs utilized to guide clinical practice parameters by the department?

    • □ Yes

    • □ No

  • 36. Please rank the ACGME core competencies according to how much they are emphasized in your journal club. Each rank may only be used once (1 = least emphasized, 6 = most emphasized).

    • □ Patient care

    • □ Medical knowledge

    • □ Practice-based learning and improvement

    • □ Interpersonal and communication skills

    • □ Professionalism

    • □ Systems-based practice

  • 37. The teaching of which core competency would you like to implement better in your journal club?

    • □ Patient care

    • □ Medical knowledge

    • □ Practice-based learning and improvement

    • □ Interpersonal and communication skills

    • □ Professionalism

    • □ Systems-based practice

  • 38. What do you think makes a successful journal club in an anesthesiology residency program? (Select all that apply)

    • □ Small number of participants

    • □ Large number of participants

    • □ Offsite location (restaurant, private home, etc)

    • □ Onsite location

    • □ Complimentary food

    • □ Discussion of clinical articles

    • □ Presence of program director

    • □ Resident facilitator

    • □ Faculty facilitator

    • □ Other (please specify):__________________________________________________

  • 39. What makes your journal club a successful one?

    _______________________________________________________________________

  • 40. Which novel approaches or recent changes have you implemented to improve your journal club and their effectiveness (website module, podcasts, etc)?

    ________________________________________________________________________

  • 41. What would you like to improve about your journal club?

    ________________________________________________________________________

Abbreviations: ACMGE, Accreditation Council for Graduate Medical Education; CA, clinical anesthesia resident (first-year, second-year, third-year); CRNA, certified registered nurse anesthetist.

Appendix B. Journal Cub Features Based on Anesthesia Residency Training Program Directors’ Responses

Features and Choices	Response, % (n)a
Locationb,c
Campus only	67.1 (49)
Off-campus only	19.2 (14)
Both campus and off-campus	13.7 (10)
Frequency
Multiple times a month	12.7 (9)
Monthly	54.9 (39)
Less frequently than monthly	32.4 (23)
Time of the day
Morning	30.6 (22)
Midday	5.6 (4)
Evening	63.9 (46)
Duration
1 h	57.5 (42)
2 h	37.0 (27)
3 h	5.5 (4)
Complementary food available
Yes	64.3 (45)
No	35.7 (25)
Formal method of appraising articles before JC
Yes (eg, reading guide)	37.5 (27)
No	62.5 (45)
Formal JC evaluation method available
Yes	40.3 (29)
No	59.7 (43)
Goals set at the beginning of the year
Yes	47.9 (34)
No	52.1 (37)
Time of article selection
Beginning of the year	1.4 (1)
Week/month before next meeting	98.6 (70)
Organizer
Residents	1.4 (1)
Faculty	44.4 (32)
Residents with faculty	54.2 (39)
Moderator
Residents	1.4 (1)
Faculty	69.9 (51)
Residents with faculty	28.8 (21)
Presenter of the articles
Residents	11.1 (8)
Faculty	5.6 (4)
Residents with faculty guidance	83.3 (60)
Chooser of the articlesb
Residents	1.4 (1)
Faculty	50.0 (36)
Residents with faculty	48.6 (35)
No. of articles presented
1–2	60.3 (44)
3–4	37.0 (27)
5–6	2.7 (2)
Average resident attendance, %d
≤25	13.9 (10)
25.1–50	34.7 (25)
50.1–75	43.1 (31)
>75	8.3 (6)
Mandatory residence attendance
Yes	69.4 (50)
No	30.6 (22)
Recorded residence attendance
Yes	88.9 (64)
No	11.1 (8)
Faculty attendance
<5	48.6 (35)
5–10	40.3 (29)
11–15	11.1 (8)
Importance rating of JC as part of the residency training by the anesthesia residency program director
Very important	32.4 (23)
Important	50.7 (36)
Somewhat important	15.5 (11)
Not important	1.4 (1)
Importance rating of JC as a part of the residency training by the residents as perceived by the anesthesia residency program director
Very important	5.6 (4)
Important	36.1 (26)
Somewhat important	50.0 (36)
Not important	8.3 (6)

Abbreviation: JC, journal club.

^a Value of n varied due to the different response rate to different questions.

^b Respondents could select more than one answer.

^c Off-campus, eg, restaurant or home.

^d Resident attendance was defined as the ratio between the average number of anesthesia residents attending a JC and total number of residents enrolled in the program.