Effects of a Flipped Classroom Curriculum on Inpatient Cardiology Resident Education

Abstract

Background

The flipped classroom is designed to reinvigorate education and utilizes “at-home” time to learn concepts and “in-class” time for clinical application. While some studies have shown positive effects of the flipped classroom in undergraduate medical education, there is a paucity of data on its use in graduate medical education.

Objective

We hypothesized that a flipped classroom curriculum of Medical Knowledge Self-Assessment Program (MKSAP) content with group case discussions could improve resident knowledge and preparedness in cardiology.

Methods

Ninety-eight internal medicine residents who rotated on the inpatient cardiology service from March to October 2017 were quasi-randomized into control and intervention groups, with the intervention group assigned MKSAP readings and cases to review on their own, accompanied by weekly case discussion. Pre-post surveys evaluated for change in knowledge and preparedness, quantity of teaching received, and use of MKSAP.

Results

A total of 93 of 98 residents (95%) participated in the curriculum. There were 37 of 51 residents (73%) in the control group and 37 of 47 residents (79%) in the intervention group who responded to pre-post assessments. In paired analysis, knowledge score did not improve significantly between the groups, nor did self-reported preparedness, number of teaching sessions per week, or reported MKSAP use. However, all participants had positive perceptions of the curriculum, and the majority felt it should be continued.

Conclusions

This flipped classroom curriculum did not affect knowledge, preparedness, or number of teaching sessions for internal medicine residents on a cardiology rotation when compared to usual teaching, although residents experiencing the new model expressed high satisfaction.

What was known and gap

Some studies have shown positive effects of the flipped classroom in undergraduate medical education, but there are few data on its use in graduate medical education.

What is new

A study comparing 2 inpatient rotations, one with a flipped classroom curriculum and one without a formal curriculum.

Limitations

Single site, single specialty limits generalizability; survey tool lacked validity evidence.

Bottom line

Implementing a standardized flipped classroom curriculum did not affect knowledge, preparedness, or number of teaching sessions for internal medicine residents on inpatient cardiology rotations compared with the usual teaching without a standardized curriculum.

Introduction

The flipped classroom, which involves learners studying core concepts on their own time and then using in-class time for application of knowledge, has become more common in undergraduate education (UME), yet it is less studied in graduate medical education (GME).¹

There is evidence to support the use of the flipped classroom in other settings, including graduate school and health professions education.^2–5 To date, there is a paucity of data in GME literature showing its effectiveness,^6–9 as most GME studies report only learner satisfaction.^10–12

To our knowledge, no published studies have attempted to enhance teaching on core subspecialty internal medicine (IM) rotations using a flipped classroom. While subspecialty rotations offer an in-depth experience for IM residents, patient acuity, volume, and competing responsibilities vary, and they may offer an inconsistent learning experience. Additionally, large academic tertiary care centers provide care to acutely ill and medically complex patients, which limits exposure to common subspecialty topics.

We hypothesized that we could create and implement a 2-part flipped classroom inpatient cardiology curriculum to fill a gap in the current curriculum, to improve residents' knowledge and preparedness with common cardiology topics, and to increase the number of teaching sessions among faculty and fellows.

Methods

Study Design

A pre-post study design was used to assess the effectiveness of the curriculum through electronic surveys managed with electronic data capture at the University of Pittsburgh Medical Center.

Participants and Settings

The study took place from March to October 2017 at the University of Pittsburgh Medical Center, a large academic hospital. All 98 residents who were assigned to the inpatient general cardiology or cardiac intensive care unit over the study period were invited to participate in the study.

Curriculum Development

A needs assessment survey was sent via REDCap to 52 cardiology faculty and fellows who regularly work with residents on the inpatient rotations. Using a Likert scale, they ranked 12 cardiology topics by importance for resident education and resident competence in handling a given topic. Based on these data, 4 topics were chosen for the curriculum: treating an ST elevation myocardial infarction and a non-ST elevation myocardial infarction, treating congestive heart failure, starting inotropic medications, and interpreting right heart catheterization data. Attending physicians also self-reported their teaching sessions, which ranged from 0 to 4 per week, with 33% (6 of 18) of faculty reporting no additional teaching outside of rounds. Residents did not participate in the needs assessment, but their input was obtained from 2 years of course evaluations in which residents requested standardized teaching and noted dissatisfaction with a lecture-based curriculum.

Intervention Versus Control Curriculum

The intervention's flipped classroom curriculum paired Medical Knowledge Self-Assessment Program (MKSAP) study books and questions with fellow- and faculty-developed cases. The cases focused on the 4 topics from the needs assessment. Content review was completed on the residents' own time by reading assigned pages in the MKSAP cardiology book.

The “in-class” portion for the intervention arm was a weekly patient case written by cardiology fellows and senior cardiology faculty using previously developed problem-based learning case templates.¹³ Each case began with a common cardiology scenario and provided residents opportunities to evaluate, diagnose, and treat the patient by utilizing content from the prereading, along with evidence-based primary literature.

The intervention curriculum included the presurvey and postsurvey, and distribution of the assigned MKSAP readings and cases. The curriculum was e-mailed to all residents on the first day of their rotation, with a week-by-week guide of the expected readings and the accompanying case. Residents were instructed to prepare for the faculty sessions during clinical downtime or after work if necessary. The faculty and fellows received a weekly associated facilitator guide with instructions on how to use the study curriculum to enhance the number of usual teaching sessions. Instructions recommended they review the case themselves and choose a day to discuss it with the residents. A time and date were not specified to allow for flexibility, given unpredictable clinical responsibilities.

The curriculum was implemented over eight 4-week resident rotations. The control and intervention groups alternated monthly. Resident scheduling was predetermined prior to the implementation and pseudorandomized, as there was no reason why a resident is assigned to a given rotation.

During the control rotations, the residents received the presurvey and postsurvey and 2 reminder e-mails. Residents in the control group did not receive the flipped classroom curriculum. They were not given the cases, nor the assigned MKSAP readings, and the faculty and fellows received no specific teaching instructions. They instead could teach how they normally would: on rounds, in stand-alone teaching sessions, or without any additional teaching outside of rounds, as there was no standardized curriculum in place for the cardiology rotations.

Outcomes

We evaluated the change in resident knowledge, attitudes, and preparedness by administering surveys prior to and immediately after the learners' 4-week cardiology rotation. Knowledge was measured as total number correct out of 20 MKSAP cardiology questions. MKSAP was used with the permission of the copyright office of the American College of Physicians. We reviewed and narrowed down approximately 200 MSKAP cardiology questions by their relationship to the 4 categories of the curriculum. The final 20 were chosen by the authors, who are medical educators in both cardiology and IM. The average percent correct of the 20 questions as reported by MKSAP was 60%. The attitude-related items pertained to the residents' expectation and enjoyment of faculty teaching and were measured using a 5-point Likert scale (1, strongly disagree, to 5, strongly agree). Preparedness for handling common cardiology diagnoses was similarly measured with a 5-point Likert scale (1, strongly disagree, to 5, strongly agree).

Resident, faculty, and fellow satisfaction in the intervention group was assessed in the postsurvey by inquiring if they felt the flipped classroom curriculum should be continued, and with free response opportunities to comment on strengths, weaknesses, and recommendations for the curriculum. All questions for presurvey and postsurvey were developed through an iterative process with expert input from our institution's medical education committee and were piloted with medical education faculty for clarity, timing, and congruence with content.

The study was approved by the University of Pittsburgh's Institutional Review Board as an exempt study.

Analysis

Demographic information was analyzed with Fisher's exact tests. Knowledge, attitudes, and preparedness of residents from matched presurveys and postsurveys were analyzed using a 2-sample t test with equal variance. All statistical analyses were performed using STATA 15 (StataCorp, College Station, TX).

Results

A total of 93 of 98 potential residents (95%) participated in the curriculum, with 51 in the control group and 47 in the intervention group. A total of 74 residents (76%) completed matched presurveys and postsurveys and were used in the data analysis. Approximately 51% (38 of 74) of participants were male, and 53% (39 of 74) were postgraduate year 1 residents. There was no difference in the number of previous cardiology rotations or career interest between the groups (Table 1).

Table 1.

Demographics of Participating Residents

Demographics	Control (n = 37)	Intervention (n = 37)	P Value
Sex
Male	20	18	.82
Female	17	19
Year
PGY-1	20	19	.95
PGY-2	5	7
PGY-3	11	10
PGY-4	1	1
Rotation
Cardiac ICU	17	18	> .99
General cardiology	20	19
Previous cardiology rotations
0 or 1	22	23	> .99
2 or more	15	14
Considering a career in cardiology?
Yes	7	11	.61
No	24	20
Unsure	6	6

Abbreviations: PGY, postgraduate year; ICU, intensive care unit.

Change in Knowledge, Attitudes, and Preparedness

There was no difference in change of knowledge between the control group and the intervention group (P = .62). Attitudinal assessment on the importance and enjoyment of faculty teaching sessions showed no difference between the groups. Finally, self-reported preparedness did not differ for all cardiology topics, or for the specific topics covered in the curriculum (Table 2).

Table 2.

Change in Resident Knowledge, Attitudes, and Preparedness From Matched Precurriculum and Postcurriculum Surveys

	Control (n = 37)			Intervention (n = 37)			P Value
	Pre	Post	Change	Pre	Post	Change
Knowledge, % correcta	58	59	1	56	58	2	.62
Attitudesb
Importantb	4.14	4.05	–0.08	4.38	4.32	–0.05	.84
Enjoyc	4.16	4.14	–0.03	4.35	4.38	0.03	.69
Preparednessd
Treat an NSTEMI/STEMIe	3.73	4.14	0.41	3.97	4.27	0.30	.51
Treat CHFe	3.81	4.27	0.46	4.08	4.35	0.27	.29
Interpret RHC datae	3.14	3.95	0.81	3.27	4.16	0.89	.68
Start inotropese	2.89	3.62	0.73	3.24	3.86	0.62	.63
Identify arrhythmias	3.35	3.86	0.51	3.32	4.05	0.73	.24
Order a stress test	3.24	3.51	0.27	3.22	3.92	0.70	.05
Treat PCA patients	2.73	3.62	0.86	3.03	3.84	0.81	.76
Interpret echocardiograms	2.43	3.00	0.57	2.84	3.43	0.59	.89
Manage IABP	1.89	2.57	0.68	1.89	2.68	0.78	.66
Identify an NSTEMI/STEMI	4.03	4.19	0.16	4.14	4.41	0.27	.46
Treat valvular disease	2.84	3.35	0.51	2.97	3.49	0.51	> .99
Perform a preoperative evaluation	3.43	3.38	–0.05	3.51	3.62	0.11	.36

Abbreviations: NSTEMI, non-ST elevation myocardial infarction; STEMI, ST elevation myocardial infarction; CHF, congestive heart failure; RHC, right heart catheterization; PCA, post–cardiac arrest; IABP, intra-aortic balloon pump.

^aKnowledge scores are percent correct out of 20 MKSAP questions.

^bAttitudes of residents toward teaching were assessed using a Likert scale response (1, strongly disagree, to 5, strongly agree) to the statement: “It is important for the faculty and fellows to teach outside of rounds.”

^cAttitudes of residents toward teaching were assessed using a Likert scale response (1, strongly disagree, to 5, strongly agree) to the statement: “I enjoy faculty and fellow teaching sessions.”

^dPreparedness score is Likert-type scale (1, strongly disagree, to 5, strongly agree) to the statement: “I feel prepared to…”.

^eTopics covered in the flipped classroom curriculum.

Curriculum Use

The uptake of the curriculum in the interventional group was subpar, as only 35% (13 of 37) of residents reported completing 3 or 4 of the cases with the faculty or fellow, and 30% (11 of 37) did not complete any. Despite the intervention group's assigned weekly readings, only 51% (19 of 37) of residents reported using their MKSAP book. In the control group, 35% (13 of 37) of residents referenced their MKSAP books. Finally, there was no difference in the number of reported teaching sessions given by faculty or fellows between the intervention (1.33 sessions per week, range 0–2) and control group (1.35 sessions per week; range 1–3; P =  .93).

Satisfaction With the Curriculum

Residents, fellows, and faculty were satisfied with the flipped classroom curriculum, as the majority recommended the curriculum be continued at 92% (34 of 37), 89% (16 of 18), and 79% (11 of 14), respectively. Free response questions in the postsurvey allowed for all participants to give feedback on the curriculum (Table 3).

Table 3.

Feedback on Flipped Classroom Curriculum From Free Response Questions in the Postsurvey

Residents	Faculty/Fellows
Areas of strength
•“Reading through the sections prior then re-enforcing the ideas with the fellow/attendings.” •“Having set cases already prepared with questions and answers.” •“Identifying topics of learning to suggest to my [senior] resident/fellow for teaching.”	•“Structured time with bilateral expectations of teaching.” •“It's standardized so can guarantee that learners get similar curriculum from month to month.” •“Their availability and thoroughness was beyond what I would normally teach impromptu.”
Areas for improvement
•“Make it more clear to the fellows/attendings about what the didactics are.” •“Set up expectations early for the residents on these sessions.” •“It was a little difficult to coordinate.”	•“For the most part the residents read neither the cases or review material ahead of time.” •“Difficult to find a day to do it when everyone available & allow sufficient time for learners to prep.” •“I already teach from my own material.”

Discussion

In this 8-month study comparing 2 inpatient cardiology rotations, one with a flipped classroom curriculum and one without a formal curriculum, we found no difference in knowledge, attitudes, or preparedness of the learners. The flipped curriculum had limited uptake, in terms of residents completing outside reading and faculty and fellows conducting case discussions, yet both groups reported high satisfaction with the new approach. Our study fits well within the current landscape of flipped classroom studies in GME that show mixed objective results^14,15 but positive perceptions of the flipped classroom curriculum overall.^7–9,15 This raises the question of whether a flipped classroom methodology works within GME curricula.¹⁶

The flipped classroom is prevalent in GME, as a 2017 survey showed over 83% of IM residency programs use it for at least some of their residency training.¹⁷ The difficulties of the flipped classroom in GME, compared with UME, fall mainly on the learner. While UME students expect assigned readings, lectures, and problem sets, GME learners expect to do more “on the job” learning with less accountability.¹⁶ Residents are taught to prioritize patients; thus, when tasked with additional responsibilities, conflict can arise, especially at this time when wellness and learning are in a delicate balance. This conflict can lead to learners not completing the “at-home” portion due to time constraints or lack of deemed importance.^14,16 Low completion rates have been noted in previous studies¹⁸ and proved true in our study, as only 35% of the participants completed at least 3 of the cases, and only 51% completed independent assigned readings.

The free response section of our survey highlighted multiple barriers to successful implementation of the flipped classroom. Main barriers included buy-in from the learners and the faculty, as well as scheduling difficulties. Faculty noted that when the learners did not prepare appropriately, cases were too long to complete in each teaching session. Given the high clinical acuity of the intensive care unit, the curriculum may be better suited to a consult or ambulatory rotation where varying schedules allow for more impromptu teaching or reading of assigned material between treating patients.

Our study has several limitations, including its implementation at a single large academic center, which may limit its generalizability. Additionally, our attitudinal and preparedness questions lacked validity evidence and may not have been interpreted by respondents as intended. Finally, the low rate of curriculum completion in the intervention group raises the question of whether the outcomes of this study are due to low uptake or curricular design, which limits the interpretation of the results.

The flipped classroom has been implemented in many domains; however, final conclusions about the efficacy of the flipped classroom in GME should be further evaluated. The next step for this curriculum will be implementation into the cardiology consult service, which inherently allows for more flexibility in the teaching schedule and for more seamless integration of education into clinical duties.

Conclusion

Implementing a standardized flipped classroom curriculum did not affect knowledge, preparedness, or number of teaching sessions for IM residents on inpatient cardiology rotations compared with usual teaching on the rotations that do not have a standardized curriculum.