Abstract
Background
The academic half-day (AHD) curriculum is an alternative to the traditional noon conference in graduate medical education, yet little is known regarding its effect on knowledge acquisition and resident satisfaction.
Objective
We investigated the association between the 2 approaches for delivering the curriculum and knowledge acquisition, as reflected by the Internal Medicine In-Training Examination (IM-ITE) scores and assessed resident learning satisfaction under both curricula.
Methods
The Cleveland Clinic Internal Medicine Residency Program transitioned from the noon conference to the AHD curriculum in 2011. Covariates for residents enrolled from 2004 to 2011 were age; sex; type of medical degree; United States Medical Licensing Examination Step 1, 2 Clinical Knowledge; and IM-ITE-1 scores. We performed univariable and multivariable linear regressions to investigate the association between covariates and IM-ITE-2 and IM-ITE-3 scores. Residents also were surveyed about their learning satisfaction in both curricula.
Results
Of 364 residents, 112 (31%) and 252 (69%) were exposed to the AHD and the noon conference curriculum, respectively. In multivariable analyses, the AHD curriculum was associated with higher IM-ITE-3 (regression coefficient, 4.8; 95% confidence interval 2.9–6.6) scores, and residents in the AHD curriculum had greater learning satisfaction compared with the noon conference cohort (Likert, 3.4 versus 3.0; P = .003).
Conclusions
The AHD curriculum was associated with improvement in resident medical knowledge acquisition and increased learner satisfaction.
What was known
The academic half-day has become an accepted alternative to the noon conference to ensure residents are exposed to didactics, but data are lacking on the effectiveness of that approach.
What is new
An academic half-day curriculum was associated with an increase in in-training examination performance in internal medicine residents and required fewer resources than the traditional noon conference.
Limitations
Single institution, observational study and retrospective data review reduce generalizability. The effect of other curriculum changes on examination performance cannot be ruled out.
Bottom line
The academic half-day appears to result in improvement in residents' medical knowledge acquisition and increased learner satisfaction.
Editor's Note: The online version (31.5KB, doc) of this article includes a diagram of resident exposure to noon conference and academic half-day teaching curricula.
Introduction
Providing meaningful learning opportunities during residency is essential for the safe and effective provision of patient care. Traditional education models include 1-hour didactic learning blocks in the morning and/or at noon. The academic half-day (AHD) is an alternative to the noon conference curriculum and involves a dedicated, weekly, 3- to 4-hour teaching period that can replace the daily 1-hour sessions.1,2 Despite being adopted in Canadian programs3 and slowly being implemented in the United States by residency programs in internal medicine and other specialties,4 the literature on the effectiveness of the AHD is limited.3–7 Currently, there is no available study, to our knowledge, directly comparing the effectiveness of the AHD to the noon conference curriculum using an objective, measureable outcome.
At the beginning of the 2011 academic year, the Cleveland Clinic Internal Medicine Residency Program implemented the AHD curriculum to increase the performance and efficiency of our teaching sessions. We sought to investigate the effectiveness of the AHD compared with the noon conference approach, hypothesizing that the AHD would be associated with improved resident medical knowledge acquisition as reflected by the performance on the internal medicine in-training examination (IM-ITE) and would be associated with higher resident satisfaction.
Methods
Study Participants and Setting
We included residents in our internal medicine program between 2004 and 2011. Residents enrolled between 2004 and 2008 experienced the noon conference approach, those enrolled after 2011 were exposed to the AHD curriculum, and residents enrolled in 2009 and 2010 had exposure to both the noon conference and the AHD approach.
In 2011, our internal medicine program expanded to include 49 residents from Huron Hospital, who trained under the same curriculum and were also included in our study. We collected the following data for all residents: age, sex, type of medical degree (doctor of medicine or doctor of osteopathy), starting residency program (Cleveland Clinic or Huron Hospital), enrollment year, time gap between medical school graduation and residency enrollment (GME gap), United States Medical Licensing Examination Step 1 and 2 Clinical Knowledge (USMLE-1 and USMLE-2 CK) scores, and IM-ITE scores (percentage correct) in postgraduate years (PGY)-1, -2, and -3 (IM-ITE-1,2,3). In 2011, we started requiring all residents who scored lower than the 50th percentile on their IM-ITE to undergo a self-study program and submit written answers to the learning objectives from the missed questions on their IM-ITE, and enrollment in that intervention also was included as a covariate in this study.
We defined our primary outcome as the IM-ITE-2 and IM-ITE-3 scores to reflect knowledge acquisition under the AHD curriculum. The IM-ITE scores for residents exposed to both the noon conference and the AHD curricula (IM-ITE-3 for those who enrolled in 2009 and IM-ITE-2 for those who enrolled in 2010) were excluded from the primary outcome analyses.
The Institutional Review Board at the Cleveland Clinic deemed this study exempt.
Teaching Curriculum
From 2004 to 2010, the 12-month recurring curriculum included weekday, daily noon conferences. In 2011, we transitioned to an AHD curriculum, which included a 3-hour conference from 10:00 am to 1:00 pm, occurring once a week for each PGY class (Tuesdays for PGY-1, Wednesdays for PGY-2, and Fridays for PGY-3). It included 2 internal medicine lectures and 1 case presentation. The teaching topics were chosen in accordance to the IM-ITE Blueprint from the American College of Physicians8 and were repeated 2 times during the 3-year training period. In both models, lecturers were scheduled approximately 6 months in advance, and all residents, except for those on intensive care unit rotations, were required to attend all teaching sessions. We used an electronic card-swipe system to record the attendance. Lunch was provided for the residents attending the conferences.
Internal Medicine In-Training Examination
The IM-ITE is a validated, standardized examination aimed at assessing internal medicine residents' medical knowledge and identifying areas of deficiencies that require further learning intervention.8 In addition, it predicts resident performance on the American Board of Internal Medicine Certification Examination.9–12 We made no recommendations for our residents to study for the IM-ITE and encouraged them to use the results to guide their learning. We used the IM-ITE raw scores (percentage of questions correct) as our study endpoints.
Resident Survey
We surveyed residents in both AHD and noon conference curricula, using REDCap electronic data capture tools.13 Questions asked about the percentage of required conferences residents attended, how often they had enough time to attend them, how often they arrived to them on time, and stayed for their duration. We also assessed residents' agreeability to the teaching materials and overall satisfaction with the curriculum, adapting questions from previous literature aimed at assessing learner satisfaction.3,14–16 To the best of our knowledge, no available validated survey is available for the purpose of our study.
Statistical Analysis
We treated the variables of sex, medical degree, starting program, and self-study enrollment as categorical variables, and age, starting year, GME gap, USMLE-1, USMLE-2 CK, IM-ITE-1, IM-ITE-2, and IM-ITE-3 scores as continuous variables. We tested for associations among categorical variables using the Pearson χ2 test or Fisher exact test as appropriate and for associations among continuous variables using the Wilcoxon-Mann-Whitney test. We performed univariable and multivariable linear regressions on the testing variables. We used regression coefficients and 95% confidence intervals (CI) to describe the magnitude and directionality of the association between covariates and IM-ITE-2 and IM-ITE-3 scores. We recorded survey responses on Likert ordinal scales, summarizing them as means, and used the Wilcoxon-Mann-Whitney method to test for differences between responses from AHD and noon conference residents. Statistical significance was defined as P < .05. Data preparation, statistical analysis, and graphic generation were performed using the open source statistical package R-2.14.2 (R Core Team, 2012).
Results
Thirty-one percent (112 of 364) of residents trained in the AHD curriculum and 69% (252 of 364) had the noon conference curriculum. Residents were similar in age and had comparable USMLE-1, USMLE-2 CK, and IM-ITE-1 scores. The AHD cohort had more women and fewer doctor of osteopathy residents, and one-third of the cohort was made up of residents who transferred from Huron Hospital. There were 39 residents (35%) in the AHD format who enrolled in the self-study program. Residents in the AHD curriculum obtained higher IM-ITE-2 (65.1 versus 62.9, P = .015) and IM-ITE-3 (69.8 versus 66.7, P = .006) scores compared with residents in the noon conference curriculum (table 1). Fifty-seven residents enrolled in 2009, and 56 had exposure to both the noon conference and the AHD format in 2010. Their IM-ITE scores were, therefore, excluded from the primary analyses.
TABLE 1.
Baseline Resident Characteristics
In univariable analyses, the AHD curriculum was associated with higher IM-ITE-3 scores, but not with higher IM-ITE-2 scores, compared with the noon conference approach. In addition, sex, GME gap, USMLE-1, USMLE-2 CK, and IM-ITE-1 scores and starting residency program were associated with IM-ITE-2, IM-ITE-3, or both scores (table 2). Enrollment in the self-study program was not associated with an increase in IM-ITE-2 or IM-ITE-3 scores. There was no statistical significance for IM-ITE-2 scores with enrollment year (Figure) before the introduction of AHD. That variable was, therefore, included in the model to adjust for possible temporal effect.
TABLE 2.
P Values From Univariable Analyses of Association Among Resident Characteristics and Internal Medicine In-Training Examination (IM-ITE) Scores
FIGURE .
Internal Medicine In-Training Examination (IM-ITE) Trends Between 2004 and 2009
a Statistically significant, included in multivariable analyses.
In multivariable analyses, AHD curriculum was positively associated with the IM-ITE-3 scores (coefficient = 4.8; 95% CI 2.9–6.6; P = < .001), suggesting that it was effective in helping residents acquire new medical knowledge. In addition, performances on the USMLE-1 and IM-ITE-1 were positively associated with IM-ITE-2 and IM-ITE-3 scores (table 3).
TABLE 3.
Multivariable Analyses of Association With Internal Medicine In-Training Examination (IM-ITE) Scores
Our postintervention survey response rate was 73% (57 of 78) for residents who trained in the AHD curriculum and 100% (78 of 78) for those who studied in the noon conference approach. Residents in the AHD curriculum reported attending more of the required teaching sessions, having adequate time to get to them more often, arriving to them on time, and staying for them longer. They were also more agreeable to the teaching materials and had higher satisfaction with the teaching curriculum compared with residents in the noon conference curriculum (table 4).
TABLE 4.
Survey of Residents in Academic Half-Day (AHD) and Noon Conference Curricula
To confirm the reported increase in conference attendance, we reviewed the attendance records for the academic years pre-AHD and post-AHD. There was a 6% absolute increase in attendance in the AHD curriculum compared with the noon conference.
Discussion
Our findings show that the AHD curriculum was associated with a better performance on the IM-ITE-3 compared with a noon conference curriculum. In addition, residents in the AHD reported higher conference attendance and overall satisfaction.
The literature regarding the implementation and success of the AHD curriculum is scarce, limited to descriptions and semiquantitative analyses of data obtained from resident surveys.3,5,6 One study7 found that resident attendance at didactic lectures in a block conference format did not improve long-term knowledge. A description of the AHD curriculum at 3 different internal medicine residency programs suggests a possible association with improvement in IM-ITE scores; however, it was limited to an analysis of the change in the raw IM-ITE scores before and after the introduction of the AHD curriculum.4 In addition, at 1 of the programs, multiple-choice testing was implemented, suggesting possible confounding.17
There are several ways in which our AHD curriculum could improve medical knowledge acquisition. First, we made it the responsibility of the whole team that residents attend their respective AHD sessions. We instructed residents to forward their pagers to their team members. That likely reduced interruptions and allowed residents to focus their attention on learning. Second, the AHD model provided a focused curriculum for each PGY level, with teaching material tailored to an individual class to enhance learning. Third, smaller groups of residents allowed for more interactive sessions. The presence of only PGY peers may have provided a safer learning environment, allowing learners to freely express themselves, which is a key component of adult learning theory.18
Our results showed an improvement of 4.8% (range, 2.9%–6.6%) in the IM-ITE raw percentage scores when training under the AHD curriculum. A previous study19 reported raw score improvements of 3.9% with conference attendance and 4.5% with self-directed reading, and a multiple-choice testing program coupled with a yearlong elective experience resulted in improved IM-ITE scores of 5% to 6%.17 Previous literature also describes conference attendance to be positively associated with performance on IM-ITE,19,20 and the lack of disruptions likely also played a significant role in facilitating medical knowledge acquisition. It is important to note that the amount of exposure to dedicated teaching is less in the AHD compared with the noon conference model (3 hours versus 5 hours), yet the magnitude of improvement in scores is similar to interventions that require additional dedicated time. The AHD model also increased resident satisfaction, similar to findings from a study of the implementation of an AHD curriculum in an orthopedic residency program.21
The self-study program introduced in 2011 for the residents who scored lower than the 50th percentile on the IM-ITEs could have improved IM-ITE performance. However, our findings did not support that hypothesis.
Our observational study is limited by its retrospective nature. In addition, although we included some of the variables previously identified as associated with IM-ITE performance, we were not able to collect data regarding the use of the electronic medical resource UpToDate, quality of life, emotional exhaustion, and educational debt, which also have been associated with medical knowledge acquisition.19,22 Finally, small curriculum changes that took place during the study period could not be factored into our analysis, which could have influenced the results.
We think it is feasible to implement the AHD curriculum at other internal medicine residency programs. It requires 6 lecturers per week (2 lecturers for each PGY class). Residents are excused from clinical duties for 3 hours, and their work is covered by other team members. Scheduling is done by chief residents and does not require additional support.
Conclusion
The transition from the traditional noon conference to an AHD curriculum was associated with improved internal medicine knowledge acquisition and higher resident learning satisfaction. Internal medicine residency programs may consider adopting the AHD curriculum as an addition or alternative to the noon conference for graduate medical education.
Footnotes
All authors are at the Cleveland Clinic. Duc Ha, MD, is Medical Resident, Department of Internal Medicine; Michael Faulx, MD, is Associate Program Director, Department of Internal Medicine; Carlos Isada, MD, is Associate Program Director, Department of Internal Medicine; Michael Kattan, PhD, is Department Chair, Department of Quantitative Health Sciences; Changhong Yu, MS, is Biostatician, Department of Quantitative Health Sciences; Jeff Olender, MS, is Administrator, Department of Internal Medicine; Craig Nielsen, MD, is Program Director, Department of Internal Medicine; and Andrei Brateanu, MD, is Associate Program Director, Department of Internal Medicine.
Funding: The authors report no external funding source for this study.
References
- 1.Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298(9):1023–1037. doi: 10.1001/jama.298.9.1023. [DOI] [PubMed] [Google Scholar]
- 2.Zweifler J, Ringel M, Maudlin RK, Blossom HJ. Extended educational sessions at three family medicine residency programs. Acad Med. 1996;71(10):1059–1063. doi: 10.1097/00001888-199610000-00011. [DOI] [PubMed] [Google Scholar]
- 3.Chalk C. The academic half-day in Canadian neurology residency programs. Can J Neurol Sci. 2004;31(4):511–513. doi: 10.1017/s0317167100003735. [DOI] [PubMed] [Google Scholar]
- 4.Batalden MK, Warm EJ, Logio LS. Beyond a curricular design of convenience: replacing the noon conference with an academic half day in three internal medicine residency programs. Acad Med. 2013;88(5):644–651. doi: 10.1097/ACM.0b013e31828b09f4. [DOI] [PubMed] [Google Scholar]
- 5.Klein D, Schipper S. Family medicine curriculum: improving the quality of academic sessions. Can Fam Physician. 2008;54(2):214–218. [PMC free article] [PubMed] [Google Scholar]
- 6.Wong RY, Roberts JM. Real time curriculum map for internal medicine residency. BMC Med Educ. 2007;7:42. doi: 10.1186/1472-6920-7-42. doi:10.1186/1472-6920-7-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Winter RO, Picciano A, Birnberg B, Chae M, Chae S, Jacks M, et al. Resident knowledge acquisition during a block conference series. Fam Med. 2007;39(7):498–503. [PubMed] [Google Scholar]
- 8.American College of Physicians. ACP Internal Medicine In-Training Examination. http://www.acponline.org/education_recertification/education/in_training/. Updated 2013. Accessed January 15, 2014. [Google Scholar]
- 9.Grossman RS, Fincher RM, Layne RD, Seelig CB, Berkowitz LR, Levine MA. Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores. J Gen Intern Med. 1992;7(1):63–67. doi: 10.1007/BF02599105. [DOI] [PubMed] [Google Scholar]
- 10.Rollins LK, Martindale JR, Edmond M, Manser T, Scheld WM. Predicting pass rates on the American Board of Internal Medicine certifying examination. J Gen Intern Med. 1998;13(6):414–416. doi: 10.1046/j.1525-1497.1998.00122.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Waxman H, Braunstein G, Dantzker D, Goldberg S, Lefrak S, Lichstein E, et al. Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination. J Gen Intern Med. 1994;9(12):692–694. doi: 10.1007/BF02599012. [DOI] [PubMed] [Google Scholar]
- 12.Brateanu A, Yu C, Kattan MW, Olender J, Nielsen C. A nomogram to predict the probability of passing the American Board of Internal Medicine examination. Med Educ Online. 2012;17:18810. doi: 10.3402/meo.v17i0.18810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Cannon GW, Keitz SA, Holland GJ, Chang BK, Byrne JM, Tomolo A, et al. Factors determining medical students' and residents' satisfaction during VA-based training: findings from the VA Learners' Perceptions Survey. Acad Med. 2008;83(6):611–620. doi: 10.1097/ACM.0b013e3181722e97. [DOI] [PubMed] [Google Scholar]
- 15.Chamberlain JK, Frintner MP, Melgar TA, Kaelber DC, Kan BD. Correlates and trends in training satisfaction on completion of internal medicine-pediatrics residency: a 5-year study. J Pediatr. 2012;160(4):690–696. doi: 10.1016/j.jpeds.2011.09.031. [DOI] [PubMed] [Google Scholar]
- 16.Keitz SA, Holland GJ, Melander EH, Bosworth HB, Pincus SH VA Learners' Perceptions Working Group. The Veterans Affairs Learners' Perceptions Survey: the foundation for educational quality improvement. Acad Med. 2003;78(9):910–917. doi: 10.1097/00001888-200309000-00016. [DOI] [PubMed] [Google Scholar]
- 17.Mathis BR, Warm EJ, Schauer DP, Holmboe E, Rouan GW. A multiple choice testing program coupled with a year-long elective experience is associated with improved performance on the internal medicine in-training examination. J Gen Intern Med. 2011;26(11):1253–1257. doi: 10.1007/s11606-011-1696-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kaufman DM. Applying educational theory in practice. BMJ. 2003;326(7382):213–216. doi: 10.1136/bmj.326.7382.213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22(7):962–968. doi: 10.1007/s11606-007-0206-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.McDonald FS, Zeger SL, Kolars JC. Associations of conference attendance with internal medicine in-training examination scores. Mayo Clin Proc. 2008;83(4):449–453. doi: 10.4065/83.4.449. [DOI] [PubMed] [Google Scholar]
- 21.Clair B, Chako A, Duggal N. Academic half-day: an initial survey of residents in the Harvard combined orthopaedic residency program. Orthop J (Harvard) December 2011 http://orthojournal.org/2011/12/26/academic-half-day/. Accessed September 24, 2013. [Google Scholar]
- 22.West CP, Shanafelt TD, Kolars JC. Quality of life, burnout, educational debt, and medical knowledge among internal medicine residents. JAMA. 2011;306(9):952–960. doi: 10.1001/jama.2011.1247. [DOI] [PubMed] [Google Scholar]