Abstract
Purpose
To compare the flipped classroom (home pre-taped lectures followed by in-class group exercise) to the traditional classroom (home reading assignment followed by in-class lecture) for horizontal strabismus didactics in ophthalmology residency.
Methods
All PGY2-4 residents from four US ophthalmology residencies without prior residency flipped-classroom experience were invited to esotropia and exotropia sessions sequentially, with random order and assignment to flipped and traditional classrooms. Content test scores before and after the two classrooms were compared. Surveys were administered to assess participant experience.
Results
A total of 40 residents attended each session. Likert scale evaluation of preparatory material and classroom activity did not differ between sessions; however, divided by year of training, 70% of senior residents (PGY3-4) and 39% of first-year (PGY2) residents preferred the flipped classroom over the traditional classroom. Pre- and post-test scores for the flipped classroom exceeded those of the traditional classroom for the exotropia course (P = 0.01 and P = 0.001, resp.) but not for the esotropia course. There was significant improvement between pre- and post-tests for both styles of learning.
Conclusions
The flipped classroom had a favorable effect on test scores for only one of the two strabismus subjects but was preferred over the traditional classroom among PGY3-4 residents.
Ophthalmic education in medical school is increasingly sparse and often absent.1 New strategies to enhance the efficiency and efficacy of ophthalmology residency didactics deserve attention. The flipped-classroom technique is an approach to adult learning that encourages active participation. Students watch prerecorded lectures on their own time before entering the classroom. During the instructional classroom time, students participate in small-group learning exercises rather than passively observing a lecture. Faculty are present to facilitate group discussion and to answer questions. The flipped classroom technique is overwhelmingly supported by education research in undergraduate science, engineering, and mathematics.2-5 One meta-analysis of 225 studies found a consistent advantage of the flipped-classroom technique in both test scores (average 6% improvement) and failure rates (average 33% lower) across varying class sizes, educator skill, course level, and course type.6
Problem-based learning approaches in medical schools have well-established efficacy and are now widely employed.7,8 Graduate medical education has been slow to adopt these educational techniques, possibly because the evidence for the efficacy of this approach in resident education is lacking. Only a handful of studies have been reported in few specialties, and these demonstrated favorable although modest advantages of the flipped-classroom technique over traditional lectures for residency teaching in emergency medicine,9-11 anesthesia,12 and critical care.13 This multicenter randomized pilot study compares ophthalmology residents’ knowledge acquisition and preferences for learning basic strabismus topics in traditional or flipped classrooms.
Subjects and Methods
Residents from all 3 years (PGY-2, PGY-3, and PGY-4) of ophthalmology residency at the University of Washington, Duke University, the University of Oklahoma, and the University of California at Davis were invited to participate. No participating faculty or departments had prior flipped-classroom experience. Institutional review board exemption was approved at all institutions except the University of Oklahoma, where full approval was required and obtained.
Study Materials
The attending physician authors (MTC, LBE, TLY, NGG) collaborated to develop standardized curricula for esotropia and exotropia, and each delivered both an esotropia and exotropia didactic session to the residents at their own institutions. The order and format (traditional or flipped-classroom) of the two different topics and two different styles were randomly assigned, such that each physician delivered one esotropia and one exotropia session and one flipped-classroom and one traditional lecture (see Figure 1). The two sessions were delivered in 1-hour increments 1-3 weeks apart. For the traditional classroom lecture, each author developed her own lecture materials that conformed to the curricula. The authors collaborated to develop a 30-minute slide presentation for both esotropia and exotropia. Each presenter used the presentation software’s voiceover feature to record the topic they were randomly assigned to deliver the same lecture in the flipped-classroom approach. The authors also collaborated to develop small-group activities for both esotropia and exotropia: short case presentations with photographs and open-ended questions (without overlapping cases or questions to the slide presentations) to be used in the flipped-classroom presentation on Prezi (Prezi, Inc, San Francisco, CA), an online presentation software that uses a maplike overview with zooming visual effects when advancing between slides: this style has been shown to be more engaging than traditional slide presentations.14 The Prezi activities were presented to senior residents and pediatric ophthalmology fellows at each of the institutions; none of these trainees participated in the current study, but their feedback was used to improve the Prezi presentations prior to the study proper. The esotropia and exotropia in-class Prezi activities slightly differed because of variation in curriculum development: the esotropia activity included fewer follow-up answers, whereas in the exotropia activity all questions were subsequently answered within the Prezi before proceeding to the next question. Following both classes, all residents were provided with the answer keys to the Prezi questions as an email attachment. The authors also developed 5-question Ophthalmic Knowledge Assessment Program-style questions for both esotropia and exotropia to be used as pre- and post-tests.
FIG 1.
Schematic showing the randomization of each participating institution to topic (esotropia and exotropia) as well as classroom style (flipped classroom and traditional lecture). The order of classes is also shown, with the top panel representing the first class and the bottom panel representing the second class. DMEI, Dean McGee Eye Institute at University of Oklahoma; Duke, Duke University; ET, esotropia; UCD, University of California at Davis; UW, University of Washington, Seattle. XT, exotropia.
Learner Preparation and Testing
Three days prior to the classroom session for all topics and styles of teaching, residents received instructions on preparing for the session and were encouraged to attend punctually. Residents preparing for the flipped-classroom session were instructed to download and watch the 30-minute prerecorded video including voiceover on the appropriate topic. Residents preparing for the traditional classroom session were instructed to read relevant chapters from the Basic and Clinical Science Course (BCSC, American Academy of Ophthalmology) textbook. UC Davis and University of Oklahoma deviated from the protocol: UC Davis failed to ask residents to read BCSC prior to the traditional classroom, and University of Oklahoma asked all residents to read BCSC prior to both flipped and traditional classrooms. At the beginning and end of each classroom session for both topics and both styles, residents were asked to complete an anonymous pretest and identical post-test on the relevant topic. Residents had 10 minutes to complete each test.
The Flipped Classroom
After the pretest, residents were asked to form groups of 2-3, preferably including both junior and senior residents. Any medical students present were asked to join a group of at least 2 residents. One resident per group was given an email link to the online presentation created in Prezi to access on their laptop in the classroom. Residents were instructed to use the next 25 minutes to work through the Prezi as a group, discussing each case and answering the relevant questions before advancing to the next slide. Each resident was provided a paper worksheet with the same questions as on the Prezi to write answers as they proceeded through the cases. These written answers were an aid to organization and preparation for individualized discussion with the faculty instructor, but they were not analyzed for this study. The Prezi included suggested times for each case to give residents a sense of how quickly they should proceed in order to complete all cases in the allotted time. During the study session, the faculty instructor answered resident questions and discussed answers. Following 25 minutes of group work with the Prezi, the post-test was given, followed by a 9-minute interactive group discussion, led by the faculty instructor, to review answers to the post-test and key concepts from the small-group activity.
The Traditional Classroom
After the pretest, residents were given a traditional in-person lecture via presentation software for 34 minutes by the instructor and author from that institution (MTC, LBE, TLY, or NGG). Each attending delivered her lecture in her standard style. For all authors in this study, “traditional” lecture style included engaging the audience with questions and discussion. Following the lecture, the post-test was administered.
Surveys
The last minute of each one-hour session for all classes was devoted to a Likert-scale written survey for resident participants specific to either the traditional classroom or the flipped classroom, including questions about the preparatory work for that day’s session. Following completion of both sessions, residents were sent a link to an additional online survey. Residents at each institution were offered a free dinner for 100% participation in the online survey. Residents were asked to participate in all surveys even if they did not attend the sessions. All surveys were administered anonymously.
Statistical Analysis
The primary outcome measure was whether the residents favored the style of the flipped classroom over the traditional classroom, as measured by one survey question on the online survey analyzed only among residents who attended both classroom styles as well as by comparing additional Likert-style written survey questions rating their educational experience between classroom styles for both preparation and classroom work. Two slightly differently worded Likert-style questions regarding each of preparation and classroom work were included, a Cronbach’s α statistic was used to test for internal consistency between the two questions, and the average of the two was used for the final analysis. The secondary outcome measure was whether the flipped-classroom style enhanced knowledge based on pre- and post-test knowledge assessment scores compared to the traditional classroom style. Validation and internal consistency of the pre- and post-test questions were assessed by Pearson correlation (r = 0.391, P = 0.011) and Cronbach’s α (0.60). Pretest scores were compared between classroom styles to reflect the efficacy of preparation, an average difference between pre- and post-test scores was compared between the two classroom styles to reflect efficacy of the classroom style itself, and average post-test scores were also compared to understand overall effect. Chi-square testing was performed for all analysis. Additional subanalyses were performed for all outcome measures to confirm consistency of results across institutions, topic, year of residency, and order of classroom. A χ2 test was used for all comparisons of categorical data, and a t test and ANOVA were used for all continuous data. A two-sided P value of <0.05 was considered statistically significant. All analyses used SPSS V24 (IBM Corporation, New York, NY).
Results
There were 40 participants each for the flipped and for the traditional classes. A total of 41 residents of the 4 residency programs responded to the online survey following both classes, including 13 (32%) first-years (PGY2), 14 (34%) second-years, and 14 (34%) third-years. Of these, 33 (81%) attended the classes in both formats; therefore, the number of online survey participants exceeded the number of individual classroom participants.
Classroom Effectiveness
Immediately following each classroom, residents were asked to rate the effectiveness of each teaching style on a 5-point Likert scale, with 1 being least effective. These questions were validated with a Cronbach α statistic of 0.97 for the esotropia class and 0.99 for the exotropia class. Of note, all scores were high, with 92.5% of residents selecting agree or strongly agree that the flipped classroom was effective compared to 97.5% for the traditional classroom. There were no significant differences in scores between the flipped classroom and traditional classrooms overall (P = 0.28) or between those who had prepared 50% or more versus 25% or less (P = 0.19). Excluding UC Davis, where no reading assignment was assigned for the traditional classroom, residents had no preference for traditional or flipped classrooms for either the preparation or the classroom activity in both exotropia and esotropia topics. Among those who attended both classrooms, 18 of 33 (55%) preferred the flipped-classroom style overall. However, analyzed by year of training, 14 of 20 senior residents preferred the flipped-classroom style compared to 5 of 13 junior residents (70% vs 39%; P = 0.07). Of residents who attended both classroom styles, only 1 resident responded that the flipped-classroom style should never be used. Most residents (28/33 [85%]) felt that 25% or 50% of didactics teaching should be conducted in the flipped-classroom style. At least 67% of ophthalmology residents agreed or strongly agreed that the flipped-classroom approach would be appropriate for all subspecialty subjects in ophthalmology, with a preference for pediatric ophthalmology, neuro-ophthalmology, and optics (P < 0.001). See Table 1. A summary of resident comments regarding advantages and disadvantages of the flipped-classroom style is provided in Table 2.
Table 1.
Resident perspective on the flipped-classroom format for teaching various ophthalmology topics
| Topic (n = 41) | I think the following topics would be appropriate for the flipped- classroom format, no. (%) |
|||
|---|---|---|---|---|
| Strongly disagree |
Disagree | Agree | Strongly agree |
|
| Pediatric Ophthalmologya | 0 (0) | 2 (5) | 20 (49) | 17 (42) |
| Retina | 1 (2) | 7 (17) | 22 (54) | 9 (22) |
| Cornea | 1 (2) | 7 (17) | 24 (59) | 7 (17) |
| Glaucoma | 1 (2) | 7 (17) | 22 (54) | 9 (22) |
| Opticsa | 1 (2) | 3 (7) | 15 (37) | 19 (46) |
| Neuro-ophthalmologya | 1 (2) | 3 (7) | 20 (50) | 15 (37) |
| Oculoplastics | 1 (2) | 12 (29) | 19 (46) | 7 (17) |
| Ocular pathology | 3 (7) | 7 (17) | 21 (51) | 8 (20) |
Residents preferred the flipped-classroom style for pediatric ophthalmology, neuro-ophthalmology, and optics over other specialties (P < 0.001).
Table 2.
Resident comments regarding advantages and disadvantages of the flipped-classroom format following both classes
| Advantages of the flipped classroom formata | No. respondents |
|---|---|
| More interactive and engaging | 10 |
| Solidifies prior understanding from the preparation | 5 |
| Encourages residents to review material beforehand | 5 |
| Better application of concepts to clinical scenarios with critical thinking | 4 |
| The recorded lecture is easier to understand than reading material | 1 |
| More approachable learning environment, can ask silly questions | 4 |
| Less pimping, which is not always conducive to learning | 2 |
| Interactive format helps with retention of concepts | 2 |
| Able to watch pretaped lecture at own pace/time, pause, rewind, etc | 1 |
| Allows for a deeper understanding for senior residents | 2 |
| Cementing in weaker areas, correct errors in learning | 3 |
| More personalized learning in small groups | 1 |
| Weakest aspects of the flipped-classroom formata | |
| May not be able to cover as much information, takes longer | 6 |
| If resident did not prepare, may not get as much out of the session | 4 |
| Less prepared residents will impair their group’s learning | 2 |
| Does not match everyone’s learning style | 1 |
| Video lecture doesn’t allow for questions and answers | 1 |
| Doesn’t hold residents accountable for BCSC reading | 1 |
| Minimal guidance on covering difficult concepts | 1 |
| Lack of immediate feedback to understand if and how you were wrong | 2 |
| Variability in effectiveness depending on your group | 2 |
| Limited learning from the attending | 3 |
| Some content may best be learned a different way (such as memorization) | 1 |
| Risk of reusing the video lecture without updating | 1 |
| Sometimes the “blind leading the blind” scenarios, getting stuck | 2 |
| May not be challenging enough for higher level residents who can pick up more information more efficiently from a traditional lecture | 2 |
| Requires more preparation | 2 |
BCSC, Basic and Clinical Science Course.
Responses were summarized.
Testing
Residents in the flipped classroom scored higher on their pretest than those in the traditional classroom for the exotropia class (P = 0.01); this difference was not seen for the esotropia class (P = 0.86). See Figure 2. Similar results were seen when excluding UC Davis (P = 0.02 and P = 0.74, resp.). The residents in both the flipped and traditional classrooms made significant improvement from pre- to post-test (P = 0.04 and P < 0.001, resp.). Improvement was most apparent in the flipped classroom for exotropia, where scores improved from 3.9 ± 1.0 to 4.7 ± 0.5 (P = 0.001). The post-test scores favored the flipped-classroom style for exotropia (P = 0.001) but the traditional classroom for esotropia (P = 0.01). Similar results were seen when UC Davis was excluded (P = 0.001 and P = 0.01, resp.).
FIG 2.
Graph shows mean score out of 5 Ophthalmic Knowledge Assessment Program (OKAP)-style content test questions given immediately before (pretest) and after (post-test) the classroom to ophthalmology residents at 4 institutions. Error bars show the standard deviation. Results compare the flipped classroom to the traditional classroom for all classes, the exotropia class and the esotropia class. Both class styles demonstrated a significant improvement in test scores between pretest and post-test (P = 0.04 and P < 0.001, resp.). The post-test scores favored the flipped classroom style for exotropia (P= 0.001) but favored the traditional classroom for esotropia (P = 0.009).
Preparation for the Classroom
Because UC Davis and University of Oklahoma deviated from the study protocol, the median percent preparation was 0% (range, 0%-75%) for UC Davis compared to 75% (range, 0%-100%) for the remaining groups (P < 0.001). However, even with UC Davis excluded from the analysis, the median preparation for the flipped classroom was 100% (range, 0%-100%) compared to 75% (range, 0%-100%) for the traditional classroom (P < 0.001). Among all four programs, more residents prepared for the flipped classroom compared to the traditional classroom, with 27 of 37 (73%) reviewing 100% of the assigned material and 29 of 37 (78%) reviewing at least 75% of the assigned material for the flipped classroom (video lecture). In the traditional-classroom setting, 9 of 31 residents (29%) reviewed 100% of the assigned material (P = 0.001) and 18 of 31 (58%) reviewed at least 75% (P = 0.11). Immediately following each classroom session, residents were asked to rate the effectiveness of the preparation assignment. These questions were validated with a Cronbach α statistic of 0.98 for the esotropia class and 1.0 for the exotropia class. Of those who had completed at least 50% of the preparation assignment, 24 of 36 (67%) strongly agreed that the flipped-classroom preparatory work was effective compared to 12 of 25 (48%) for the traditional classroom (P = 0.07). The overall pretest results support greater preparation for the flipped classroom compared to the traditional classroom (average score, 3.4 ± 1.3 vs 2.8 ± 1.3 [P = 0.04]). See Figure 2.
Discussion
This randomized, multicenter study demonstrated modest preference among ophthalmology residents for the flipped classroom over the traditional classroom in learning strabismus, with more robust preference for the flipped classroom among upper level residents. Following both classes, most residents agreed that 25% or 50% of didactics teaching should be conducted in the flipped-classroom style, including pediatric ophthalmology, neuro-ophthalmology, and optics. All courses, taught in either style, demonstrated significant improvement in participants’ content test scores from pre- to post-test.
The flipped-classroom style requires that residents review a video lecture before attending the classroom session. Based on resident comments, the video lecture was particularly effective, because it allowed them to pause, repeat, and fast-forward the lecture as desired. Residents found the lecture easier to understand than the reading material and felt compelled to prepare for more active participation. Therefore, compliance with completing the preparatory assignment favored the video lecture over BCSC reading, even when excluding residents who did not specifically receive a reading assignment. Furthermore, higher pretest scores suggest that the flipped-classroom approach may be advantageous regardless of what happens in the classroom. Nonetheless, residents also remarked that the flipped-classroom format requires more preparation time; this may explain why most residents felt it should not occupy more than 50% of didactic sessions.
Many residents commented that the flipped-classroom format was more interactive and engaging, helping them to solidify preparatory work and increase understanding through repetition in a different format, including more real-life clinical case scenarios. Anecdotally, faculty noticed higher resident participation and engagement in the flipped classroom, although this was not specifically measured. Although used for the in-class flipped-classroom activity in this study, Prezi was not specifically assessed relative to other presentation software. The authors feel that other presentation formats would be reasonable alternatives. Finally, mixed groups were felt to enhance the learning experience, including at least one senior “expert” resident to reinforce prior knowledge and at least one junior “learner” resident to prompt interactive questions. It is possible that upper-level residents viewed the flipped classroom more favorably than first-year residents because they had a deeper familiarity with the subject matter and were more willing to participate during in-class discussions.
Content testing favored the flipped classroom in the exotropia but not the esotropia classes. The result could differ when testing retention of knowledge at a later date. A study in anesthesia residency found no significant difference in test scores immediately following the course; however, 4-month retention testing demonstrated a significant advantage for the flipped classroom.12 It is unknown whether improved content testing translates to better clinical performance.
Variability by topic has been reported previously, and is not surprising given that each topic has its own curriculum and test questions.11 The esotropia flipped-classroom activity only included a few embedded answers as the residents worked through questions, whereas in the exotropia flipped-classroom each question was answered before the next began. Although an answer key for both sessions was provided as an email attachment following the class, residents may have been less likely to review the answer key at a later time. Residents mentioned concern with lack of immediate feedback to understand if and how they were wrong. These findings underscore the importance of immediate feedback within the flipped-classroom in-class exercise.
This study was also limited by additional deviations from study protocol, including one site (UC Davis) providing no reading assignment prior to the traditional lecture and another (University of Oklahoma) providing BCSC reading assignments prior to both sessions. Although these changes may have affected the results, they may be more representative of real-life practice. Furthermore, significantly higher compliance in preparation, pretest content scores for both exotropia and esotropia, and post-test scores for exotropia for the flipped classroom compared to the traditional classroom were seen even with UC Davis excluded from the analysis. This study also failed to account for prior experience with the flipped-classroom format, did not include results for those who did not attend the classroom, and did not include direct pre- and post-test comparisons among individual residents. Finally, this study could not control for instructor enthusiasm while teaching in the flipped-classroom compared to the traditional classroom. Future studies should provide more consistent feedback for in-class exercises, explore the effectiveness in teaching other ophthalmology topics in a flipped-classroom style, involve more sites and lecturers, and use longer follow-up to test knowledge retention.
Acknowledgments
Funded by an unrestricted grant from Research to Prevent Blindness (New York, NY) and the NIH CORE Grant (EY001730). The funding organizations had no role in the design or conduct of this research.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Succar T, Grigg J, Beaver HA, Lee AG. A systematic review of best practices in teaching ophthalmology to medical students. Surv Ophthalmol 2016;61:83–94. [DOI] [PubMed] [Google Scholar]
- 2.Casasola T, Nguyen T, Warschauer M, Schenke K. Can flipping the classroom work? Evidence from undergraduate chemistry. Int J Teach Learn High Educ 2017;29:421–35. [Google Scholar]
- 3.Baepler PM, Walker JD, Driessen M. It’s not about seat time: blending, flipping, and efficiency in active learning classrooms. Comput Educ 2014;78:227–36. [Google Scholar]
- 4.Khoumsi A, Hadjou B. Learning probabilities in computer engineering using a competency - and a problem-based approach. J STEM Educ 2005;6:5–13. [Google Scholar]
- 5.O’Flaherty J, Phillips C. The use of flipped classrooms in higher education: a scoping review. Internet High Educ 2015;25:85–95. [Google Scholar]
- 6.Freeman S, Eddy SL, McDonough M, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A 2014;111:8410–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Soriano RP, Blatt B, Coplit L, et al. Teaching medical students how to teach: a national survey of students-as-teachers programs in U.S. medical schools. Acad Med 2010;85:1725–31. [DOI] [PubMed] [Google Scholar]
- 8.Yu TC, Wilson NC, Singh PP, Lemanu DP, Hawken SJ, Hill AG. Medical students-as-teachers: a systematic review of peer-assisted teaching during medical school. Adv Med Educ Pract 2011;2:157–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wolff M, Wagner MJ, Poznanski S, Schiller J, Santen S. Not another boring lecture: engaging learners with active learning techniques. J Emerg Med 2015;48:85–93. [DOI] [PubMed] [Google Scholar]
- 10.Young TP, Bailey CJ, Guptill M, Thorp AW, Thomas TL. The flipped classroom: a modality for mixed asynchronous and synchronous learning in a residency program. West J Emerg Med 2014;15:938–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Riddell J, Jhun P, Fung CC, et al. Does the flipped classroom improve learning in graduate medical education? J Grad Med Educ 2017;9:491–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Martinelli SM, Chen F, DiLorenzo AN, et al. Results of a flipped classroom teaching approach in anesthesiology residents. J Grad Med Educ 2017;9:485–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Marchalot A, Dureuil B, Veber B, et al. Effectiveness of a blended learning course and flipped classroom in first year anaesthesia training. Anaesth Crit Care Pain Med 2018;37:411–15. [DOI] [PubMed] [Google Scholar]
- 14.Moulton ST, Türkay S, Kosslyn SM. Does a presentation's medium affect its message? PowerPoint, Prezi, and oral presentations. PLoS One 2017;12:e0178774. [DOI] [PMC free article] [PubMed] [Google Scholar]