Flipping Content to Improve Student Examination Performance in a Pharmacogenomics Course

Abstract

Objective. To develop, implement, and evaluate active learning in a flipped class to improve student examination performance in the genetic foundations of pharmacogenomics.

Design. The flipped classroom model was adopted in which a guided-inquiry learning activity was developed and conducted to complement recorded, previously viewed didactic lectures. The activity was constructed to focus on critical thinking and application of core principles of genetic crosses and pedigree analysis. A combination of independent work and active discussion with volunteer and guided student response provided student-facilitator interaction.

Assessment. Student learning was evaluated by comparing pretest and posttest formative assessment results and by the comparison of prior years’ examination performance on a subset of content for which no flipped classroom learning activities occurred. There was no significant difference between examination scores between the flipped classroom and previous approaches. An item-by-item analysis of the content reflected a significant change in performance on questions addressed in the flipped classroom exercise.

Conclusion. The flipped class instructional model in this project included active-learning activities and formative assessments that provided students spaced and repetitive curricular engagement. The intervention transformed the classroom interactions of faculty members and students and contributed to improved student examination performance.

INTRODUCTION

The “ flipped class” moniker was derived from the inversion of what was traditionally classwork and homework. The rationale of the flipped class is to reach every student, in every class, every day.¹ In the flipped class, lectures are viewed outside of class, permitting in-class time to be spent on active-learning practices to which sutdents can apply previously viewed lecture content. The flipped class was conceived to provide students who had missed scheduled class with access to class content. Early iterations of lecture capture employed technologies such as videotape and writable compact discs. Over the past decade, digital technologies for recording and sharing educational content have expanded greatly, and the flipped class leverages these trends. Many pharmacy schools have embraced lecture capture,² which comports with the widespread consumer adoption of technologies to record and view digital content at home, on smartphones, and on tablets such as iPads.³ The flipped class instructional model leverages the availability of high quality digital video content and expanding distribution modalities including wireless mobile devices. As part of the flipped classroom, the instructor’s role must expand beyond didactic lecturer to include class activity designer of and assesser of real-time of student learning.

The use of active teaching and learning methodologies, many of which are based on cognitive science research on human learning, were codified into policy changes by the American Council for Pharmacy Education (ACPE) Accreditation Standards in 2006. The council defined active learning as a style of teaching that requires the learner to formulate answers to questions based on acquired knowledge while continuing to search for new knowledge that may provide better, more complete answers.⁴ The flipped classroom or lecture halls without lectures, was endorsed as an active, student-centered instructional approach in 2011 in the New England Journal of Medicine. In the article “ Lecture Halls without Lectures,” senior administrators from the Stanford School of Medicine and the Stanford Graduate School of Business stated “We propose embracing a flipped-classroom model, in which students absorb an instructor’s lecture in a digital format as homework, freeing up class time for a focus on applications, including emotion-provoking simulation exercises. Students would welcome more opportunities for case-based, problem-based, and team-based exercises—strategies that activate prior knowledge. Teachers would be able to actually teach, rather than merely make speeches.”⁵ Increasingly, the flipped class is the focus of projects regarding the scholarship of teaching and learning. The instructional intervention of combining assessment for learning and flipped class activities can improve academic outcomes, yet the topic requires additional study.⁶

Identifying curricular content suitable for designing class activities is central to successfully employing the flipped class instructional model. Choosing with fidelity what content to reframe as an active flipped class activity comports with Shulman’s concept of pedagogical content knowledge (PCK),⁷ a complex and multidimensional construct that comprises educators’ professional understanding and strategies for communicating that understanding. It represents a specialized body of professional knowledge that distinguishes teachers from others who might know a subject well, but who have no occasion to develop the knowledge required to teach the subject. Such knowledge includes, for the most, regularly taught topics in one’s subject area, the most useful form of representations for those topics, and the most powerful analogies, illustrations, examples, explanations, and demonstrations—in short, ways of representing and formulating a subject that make it comprehensible to others.⁸ An understanding of what makes learning specific topics easy or difficult and the conceptions/preconceptions that students of different ages and backgrounds bring with them are characteristics of PCK.⁸

Among American Association of Colleges of Pharmacy (AACP) member pharmacy schools for the 2013-2014 academic year, only 6% required genetics and 2% required molecular biology as prerequisites for admission.⁹ The site of this project, like the majority of pharmacy schools, does not require genetics or molecular biology as prerequisites for admission, even though an understanding of genetics and molecular biology is critical for applying pharmacogenomics in practice.¹⁰ For this study, curricular content was identified as a good match for the combination of video content delivered outside of class and direct instruction and informal assessment in class, both cornerstones of the flipped class instructional model. The authors hypothesized that student examination performance could be improved by incorporating content regarding pedigree analysis, genetic inheritance, and the assignment of presumptive genotypes to individuals into a flipped class activity which featured voluntary pre/posttest assessments.

DESIGN

The statistical design of the study was a between-group comparison of a subset of examination questions for spring classes in years when traditional didactic instruction took place (2012, 2013) and when the flipped class instructional model intervention occurred (2014). The school’s institutional review board approved the project.The Essentials of Pharmacogenomics is a required first professional (P1) year course in the Shenandoah University pharmacy curriculum. Taught over 15 weeks in the spring of the first year (P1), the course is broken into 4 main blocks: genetics, molecular biology, applications of genomics in pharmacy and medicine, and considerations of pharmacogenomics in practice.¹¹ The study examined student performance across 3 years (spring 2012, 2013, and 2014) in the course. Participants entering pharmacy school with a bachelor’s degree included 47% (34/72) in 2011, 66% (81/123) in 2012, and 54% (60/110) in 2013. Students who had not completed an undergraduate degree upon entering pharmacy school included 63% (46/72) in 2011, 34% (42/123) in 2012, and 38% (42/110) in 2013. To evaluate the assumption of the equality of variance among the 3 groups, the authors considered: (1) end-of-semester grades in the prerequisite class Integrated Basic Health Sciences I; (2) previous exposure to genetics and molecular biology; and (3) grade point average (GPA) of each class upon admission to pharmacy school.

Integrated Basic Health Sciences I is a 3-credit, 8-week course offered in the first semester of the P1 year and is the first module in a course series that spans biochemistry, cell and molecular biology, and anatomy and physiology. Since biochemistry and cell and molecular biology courses are not prerequisites for enrollment in the school in this study, approximately half of this module (4 weeks) is dedicated to an introduction to these topics with an emphasis on how this material will be integrated with anatomy and physiology throughout the remainder of the P1 year. The course is taught didactically using video teleconferencing technology to connect 2 campuses, with an equal number of lectures originating on each campus. Historically, students who fail to master this material presented in this course struggle in the Essentials of Pharmacogenomics course, which covers these areas in greater depth; therefore, the former is a prerequisite for the latter. A one-way analysis of variance (ANOVA) found no significant difference between students’ final average grades in the prerequisite class in 2012, 2013, and 2014 (p=0.11).

Prior to the flipped classroom intervention, students were directed to review 6 prerecorded video mini-lectures on basic genetics terminology, single-gene inheritance including autosomal and sex linked inheritance, pedigree analysis and risk calculations, 2-gene inheritance with independent assortment, meiotic recombination, and polygenic inheritance in class. The lectures were recorded in a voice-over slideshow format using Camtasia Relay (Techsmith, Lansing, MI) and were accessed by the students through the iTunes University platform (Apple, Cupertino, CA). In the next class session, students completed a voluntary pretest to assess their comprehension of the material from listening and watching the voice-over slide format lectures. The subsequent in-class activity consisted of a brief review of genetic concepts by the instructor, student participation answering instructor-posed questions, and an in-class activity to assess students’ grasp of the material presented. Within 12 hours of completion of the class session, students were requested to complete a voluntary posttest.

Students in the 2014 class participated in the flipped classroom activity, while in 2012 and 2013, the lecture material was presented as 2 live lectures during the scheduled class, recorded via Camtasia Relay, and made available for viewing on iTunes University after class. The 2 recorded lectures in 2012 and 2013, totaled 120 minutes of instructional time. No formative assessments were conducted in 2012 or 2013 and viewing recorded class lectures after class was optional. In 2014 students were requested to view 6 pre-recorded video mini-lectures, totaling 120 minutes of instructional time prior to the flipped class. The same instructor performed the lectures on the same learning objectives all 3 years. Nearly identical examination questions from this material were used in 2012, 2013, and 2014. Examinations were administered electronically using a secure browser in a proctored setting. Questionmark Perception (Perception, Norwalk, CT) was selected for the creation, delivery, and analysis of examinations. After completing the examination, students were given their scores and feedback on incorrect answers only. Examinations were not passed back to students and while no examination key was posted, it was available for viewing by appointment with the instructor.

The year of the flipped-class intervention, a pre/post-test model was used as a formative assessment to gauge student understanding and refine instructional processes. The voluntary pretest and posttest were were part of the students’ grades. The pretest and posttest were identical, and the questions were similar in concept, though not in content, to the items on the examination (ie, the genotypes and phenotypes were different on the pre/post-test than on the examination). Students were given access to their answers and the correct answer rationale after the flipped-class exercise and prior to the examination on this material. Information regarding specific question topics can be found in Appendix 1.

The flipped-class session started with a brief review of autosomal vs sex-linked genes and dominant vs recessive mutations, then moved onto autosomal and sex-linked crosses and the expected genotype and phenotype ratios (pre/post questions 1 and 2). Next, students had the opportunity to practice pedigree analysis by determining the mode of inheritance as well as assigning presumptive genotypes to individuals. Students were also asked to make predictions (risk calculation) based on the knowledge obtained from pedigrees (pre/post questions 5 and 6). Finally, students were required to determine phenotypes from genotypes and perform 2- gene crosses (pre/post questions 3 and 4). The flipped class session required students to actively fill in blanks in a presentation (eg, what is the phenotype of an individual with the genotype Aa if the disorder has a recessive mode of inheritance; is this pedigree an example of autosomal or sex-linked inheritance?) and solve problems as a class. Additionally, breaks were provided during the in-class session for students to complete an exercise containing a selection of questions posed in class. Many of these questions were similar in concept, though not in content, to questions that appeared on the pre/posttest and on the examination.

EVALUATION AND ASSESSMENT

In spring 2014, teaching strategies consistent with the flipped or inverted class model, such as active, student-centered learning activities, were employed to replace the didactic method. Student learning was measured by series of assessments including a pretest and posttest in 2014, and a comparison of student performance (2012 vs 2013 vs 2014) on the subset of examination questions relating to the flipped classroom content. Data were analyzed using IBM SPSS/PCv22 (IBM, Armonk, NY). Because previous exposure to genetics and molecular biology could influence performance in a genomics course, the participating classes’ prior exposure to genetics and molecular biology was compared. In this study, the P1 classes taking Essentials of Pharmacogenomics in spring 2012, 2013, and 2014 had similar prepharmacy school exposure to coursework in genetics (52-60%, p=0.49) and molecular biology (60-64%, p=0.82) ( Table 1).

Table 2.

Student Assessment Performance in a Pharmacogenomics Course

Table 1.

Previous Student Coursework in Genetics and Molecular Biology

Differences in GPAs scores for the P1 classes in 2011 (n=79 [3.26, SD=0.40, range 2.45-4.0]), 2012 (n=123 [3.10, SD=0.42, range 2.47-4.0]), and 2013 (n=116 [3.28, SD=0.34, range 2.51-4.0]) were significant (p<0.0001). Post hoc comparisons using the Tukey HSD test indicated that the mean score for the 2012 P1 class was significantly lower than classes in 2011 and 2013. The authors believed the practical impact of a 0.1 GPA difference should not exclude the 2012 class for comparison in this study. Based on the analysis of each class’s academic performance in the required prerequisite course, previous exposure to genetics or molecular biology, and entering GPA, the authors concluded that the 3 participating P1 classes of students (2011, 2012, and 2013) were suitable for comparison.

A paired 2-sample for means t test on students who completed both the pretest and posttest assessements (n=87 [66.2, SD=22.4, range 0-100]) demonstrated significant improvement in student performance at the completion of the flipped class (p<0.05). Descriptive statistics for the 10-question examination subset relating to the flipped class content were computed for 2012 (n=81, [77.16, SD=12.97, range=40-100%]), 2013 (n=125, [80.48, SD=16.10, range=30-100%]), and 2014 (n=113, [82.30, SD=14.88, range=30-100%]). Students’ responses subset in 2012, 2013, and 2014 yielded a Cronbach alpha measure of reliability equal to 0.757. A one-way ANOVA revealed a nonsignificant improvement on the 10-question subset between 2012, 2013, and 2014 level (p=0.061).

To further analyze student examination performance, a year-to-year analysis of student performance was conducted for each question (Table 3). Chi-square analysis was performed on each question across all 3 years, and if a significant effect was observed, individual pairwise analyses were performed. Significant improvements were seen for questions 1, 2, 5, 8, and 9 and a significant decrease in performance was seen for question 4. There was no significant change for questions 3, 6, 7, and 10.

Table 3.

Performance on Individual Examination Questions

The topic of single and multi-gene inheritance as well as pedigree analysis and risk calculation was addressed using the flipped-class instructional model in this project. Prerecorded lectures were viewed outside class, and scheduled class time was devoted exclusively to developing and evaluating student content knowledge. While the overall improvement in subset score was not significant over the 3-year period, individual questions within this subset did show significant change.

In 2012, several questions (Q1, Q2, Q4, and Q5) were identified as challenging concepts for students, and changes were made in the 2013 didactic lecture format to help address these difficulties. The result was a significant improvement in 2013 student performance on questions pertaining to the general structure of chromosomes (Q1) and performing a basic single gene cross (Q2), a change that persisted into 2014. This suggests that didactic changes alone were sufficient for students to improve their understanding of these concepts. However, the 2013 didactic changes did not have an impact on students’ ability to perform pedigree analysis (Q4) or risk calculations (Q5), and therefore a flipped-classroom activity was planned for 2014 to focus on these 2 areas.

The instructional design of this project increased student exposure to the topic by leveraging student-controlled access to digital video outside scheduled class. More importantly, students were required to apply the content in problem-based, instructor-guided class activities. Fostering critical thinking and problem solving through the application of factual information within highly contextualized scenarios acted as a high-fidelity simulation for the examination format. The project design also included repeated exposure to the content through formative assessments, which documented student knowledge prior to and after the instructional process. Like most projects that feature pre/post designs, student performance gains in 2014 between the pretest and the posttest assessments were significant.

The flipped-class activity included an overview of application of genetic concepts, including the basics of single-gene inheritance that students traditionally found easy. However, mastering these topics is required in order to apply this knowledge to more challenging topics like pedigree analysis and risk calculations. As risk calculations were an area of particular weakness in previous years (question mean of 42-47%), the emphasis of the flipped-class activity was on performing risk calculations from a pedigree, resulting in a significant increase in student performance on this question (p<0.05). This suggests that the flipped classroom was potentially effective in improving student performance on this application-based exercise, while didactic instruction fell short (2012 vs 2013 p=0.45).

Student performance on 2-gene crosses (Q8), which were included in the flipped-classroom activity, also significantly increased (p=0.02) in 2014. Additionally, the adoption of the flipped-classroom model allowed greater didactic time to be focused on other areas, such as meiotic recombination and parental and recombinant genotypes (Q9), which was also associated with a significant improvement (p=0.03) in student performance on this question in 2014. Surprisingly, students performed significantly worse on pedigree analysis (Q4) after the flip, despite its inclusion in the flipped-classroom activity. The same optional pedigree analysis worksheet had been provided to all students in 2012, 2013, and 2014 to allow additional opportunities to practice this topic outside of class, but it is possible that in 2012 and 2013, more students voluntarily completed this activity as part of examiantion preparation, whereas in 2014, students relied on the flipped-class experience alone. A future study is planned to determine if this additional exposure to pedigree analysis has an impact on student performance on this question.

DISCUSSION

This project demonstrated the importance of utilizing a faculty member’s PCK in selecting a topic that could benefit from the flipped-class intervention. The planning and design of the class activities exemplify the backward design lesson process, starting with identified outcomes and working backwards to create learning opportunities through vodcasts, formative assessments, and class activities to support those outcomes.¹² The flipped classroom instructional model used digital video technology to extend the delivery of content outside of scheduled class, while replacing didactic lecture with formative assessments and active-learning activities. Transitioning the lecture content to digital video allowed classroom practice to focus on application of video content. Algorithmic processes associated with single and multiple gene inheritance and pedigree analysis were practiced and assessed in class. The role of the instructor was transformed from lecturing and occasionally querying “How are we doing?” or “Any questions about that?” to a higher level of questioning and cognitive coaching within a specific context. In the flipped class, the instructor talked less and questioned more, the students worked more, and the interactions between student, content, and faculty members were planned. The instructional refinements enhanced the scheduled class through the addition of formative assessments of student understanding and the requirement of active student participation. The study design included elements that comport with studies that demonstrate the positive impact of active-learning practices, including the role of increased assessments and active learning on improved student examination performance.^13-15 This project detailed how instructional design and instructional practices that emphasize data-driven continuous improvement through reflective practice may contribute to improved teaching and learning outcomes.

Further research is required given the limitations in the study design. Questions remain regarding the role of and extent to which the flipped-class intervention contributed to student examination performance gains. The pre/posttests that accompanied the flipped-class activities likely contributed to improved academic outcomes, yet these assessment practices may have confounded any impact attributable to the actual flipped-class activities. The instructional design to improve academic outcomes using a combination of authentic innovations in a design experiment is a limitation of this study. Like most design experiments in authentic settings, a lack of experimental controls such as random selection, random assignment, and control for the extraneous factors, such as previous exposure to genetics and molecular biology, limited the extent to which we could attribute improvement in educational outcomes to the flipped-class design. The inability to track student participation in viewing lecture content prior to the flipped-class activity also confounded the impact of the flipped-class exercise. The 2014 class contained 113 examination takers. Seventy-five percent (85/113) of the students chose to fully participate (complete the pretest, in-class exercise, and posttest), in the flipped classroom. Match pair scores for the pre/posttests were available for 77% (87/113) of students in 2014. Additionally, no significant difference was found between the class mean (n=113; mean=82.30 [14.88]) on the examination subset and the mean of students who fully participated in the flipped classroom pretest and posttest activities.

This could in part be attributed to the fact that 96% (108/113) of students participated in at least one of the activities in the flipped class and suggests that the flipped classroom experience, whether or not students chose to fully or partially participate, was effective at engaging students. This project’s limitations impact the generalizability of the conclusions to other contexts, but the design considerations and limitations should be considered within the context of developing technological pedagogical content knowledge, improving student outcomes, and progressively refining reflective teaching practice. Although academic gains were made in the flipped-class intervention year, the flipped class was not superior to previous years in which didactic lecture was also accompanied by increased student performance. The instructional intervention of combining assessment for learning and flipped class requires additional study to tease out impact of different factors.

SUMMARY

The flipped-class instructional model required students engage with digital video vodcast lectures prior to scheduled class to gain factual information. In this project, the vodcasts were organized by topic and were shorter in length than the previous years’ recorded class lectures. The scheduled class activity used a combination of strategies, such as chunking the vodcast content into smaller segments, retrieval practice in the form of formative assessments, and guided lectures featuring problem solving and discussion, that empirically show improved learning. The combination of these factors contributed to improved student examination scores in this project.

Appendix 1.

Pre/Post question 1, Examination question 2: Mechanism of inheritance (eg, complete vs incomplete dominance) was provided along with its associated phenotype (eg, color, size, shape). Based on this information, students were required to 1) determine the genotypes of the parents based on their phenotypes, 2) perform the requested cross, and 3) determine the phenotype(s) of the offspring (F1 generation).

Pre/Post question 2, Examination question 3: Mechanism of inheritance (eg, complete vs incomplete dominance) was provided along with its associated phenotype (eg, color, size, shape). Based on this information, students were required to (1) determine the genotypes of the parents based on their phenotypes; (2) perform the requested cross; (3) determine the genotype(s) of the offspring (F1 generation); (4) perform the requested cross on this F1 individual; and (5) determine the genotype(s) of the offspring (F2 generation).

Pre/Post question 3, Examination question 8: Students were provided the genotypes for 2-gene inheritance (eg, AA BB, Aa Bb, AA Bb) of the parents and were required to determine the number of offspring of a particular genotype by applying the concept of equal segregation of alleles and independent assortment of chromosomes.

Pre/Post question 4, Examination question 7: Students were provided the genotypes for two-gene inheritance (eg, AA BB, Aa Bb, AA Bb) of the parents and were required to determine the possible genotypes by applying the concept of equal segregation of alleles and independent assortment of chromosomes.

Pre/Post question 5, Examination question 4: Students were provided a pedigree and were required to identify the mode of inheritance of the disorder (eg, autosomal vs sex-linked and dominant vs recessive).

Pre/Post question 6, Examination question 5: Students were provided a pedigree and the mechanism of inheritance of the disorder (eg, autosomal vs sex-linked and dominant vs recessive) and were required to calculate the chance that an individual had a particular genotype.