Research, Perspectives, and Recommendations on Implementing the Flipped Classroom

Abstract

Flipped or inverted classrooms have become increasingly popular, and sometimes controversial, within higher education. Many educators have touted the potential benefits of this model and initial research regarding implementation has been primarily positive. The rationale behind the flipped classroom methodology is to increase student engagement with content, increase and improve faculty contact time with students, and enhance learning. This paper presents a summary of primary literature regarding flipped classrooms, discusses concerns and unanswered questions from both a student and faculty member perspective, and offers recommendations regarding implementation.

INTRODUCTION

Changes in student demographics, the economic climate, and Internet technologies have made the contemporary educational environment different than it was even just a decade ago. As a result, many educators are examining newer models of instruction to produce successful graduates in today’s society. Although similar techniques and philosophies have existed for decades, the formal flipped or inverted classroom methodology is a contemporary approach to educational design. A simplified description of the flipped classroom is a reversal of the traditional order of content delivery (ie, lectures) and homework. Lage et al described the flipped classroom as one in which “events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa.”¹ Traditional lecture material is “offloaded” for students to learn outside of class, freeing classroom time for more enriching activities.² Flipped classrooms are not a panacea for all of higher education’s teaching woes, however when designed and implemented properly, they may offer several advantages over traditional didactic classroom environments. The ability to revisit recorded lectures, in-class application activities, and increased attention to concept application during class have been cited as popular aspects of flipped classrooms among students.³

Learning theorists advocate that instructional strategies like those used in a flipped classroom enable students to learn and retain information better than through traditional lectures.⁴ Removed from the constraints of “providing content,” instructors can add value to the classroom experience by teaching students how to reason through problems and apply information to real-life issues. Supporters of flipped classroom methodologies hypothesize that students will spend less class time browsing the Internet, daydreaming, or being distracted in other ways, and spend more time engaged in class applying information through problem-solving, case discussions, or other activities that require thoughtful processing of content. This article summarizes current literature regarding flipped classrooms, discusses the benefits and concerns from both a student and faculty perspective, and provides recommendations for faculty members considering implementing this instructional approach.

RATIONALE FOR IMPLEMENTING FLIPPED CLASSROOM MODELS

As educators begin to eschew the traditional lecture format as the preferred teaching method, a variety of reasons are appearing in the literature regarding why flipped classrooms are a better approach. These reasons primarily include alleviating academic bulimia, encouraging personal accountability, and improving student learning.

In traditional classrooms, students listen to a lecture and subsequently complete required assignments after the lecture. Although this may not be true of every student, course, or subject matter, lecturing may lure students into a ritual of memorizing and regurgitating information for upcoming examinations.⁵ Often times, rather than truly learning the clinical application of the material, students resort to memorizing the presentation, purging the information on the examination, and ultimately failing to retain the information.⁵ This subconscious student strategy otherwise known as “bulimic learning”^5-8 has been successful for many students throughout their academic careers, which is why they will often resort to it without even recognizing how detrimental it is to learning.⁷

Instructors have long bemoaned that students come to class ill-prepared, without having completed homework or reading assignments.⁹ Active learning activities, such as those in flipped classrooms, increase student accountability for class preparation and attendance.^10-12 Although not exclusive to flipped classrooms, in-class activities based on pre-assignments encourage students to complete assignments and attend class to further comprehend subject material. Implementing flipped classroom strategies can increase student perceptions that preclass activities are important and enhance in-class learning.¹³

For any educational method to be considered successful, there must be evidence that student learning is enhanced. There are numerous examples of learning successes within other fields,^2,14,15 but research on flipped classroom methodologies in pharmacy education is still in its infancy and most of the studies regarding learning have been on a small scale within a single course or instructional module. Ferreri and O’Connor’s study involving pharmacy students in a patient self-care course found a steady improvement in students’ academic scores after implementing a flipped classroom model.¹¹ The year preceding the course redesign, there were 21 As, 66 Bs, 12 Cs, and one F. After course redesign, grade distributions improved significantly (p<0.001) to 52 As, 44 Bs, four Cs and zero Fs. In their small study of a flipped classroom format for teaching a controlled substance module in a pharmacy law class, Michaud-Sacks et al revealed significant increases (p<0.0001) in overall scores on knowledge and application quiz questions compared to the previous year.¹⁶ In a basic pharmaceutics course, McLaughlin et al also showed that final examination scores (out of 200 points) were higher (p=0.001) during the year in which a flipped classroom (165.48, n=162) was used compared to the previous year in which a traditional lecture format (160.06, n=153) was used.¹⁷ Wong et al showed that compared to a control group of students (n=105) from the previous year who were in traditional class lectures for cardiac arrhythmias, first-year pharmacy students (n=101) who received instruction via a flipped teaching method had higher mean examination scores on pharmacology (89.6% vs 56.8%, p<0.001) and therapeutics (89.2% vs 73.7%, p<0.001) of cardiac arrhythmias.¹⁸ Finally, Persky and Dupuis’s 8-year retrospective study of student performance revealed an increase in clinical pharmacokinetics course grades (p<0.001) following a conversion of the course from a lecture plus active-learning model to flipped classroom type activities (team-based learning).¹⁹

Not all designs have produced overwhelmingly positive results. Everly and Cochran compared two sections of students in a gastroenterology course lecture covering acute and chronic pancreatitis and found no significant differences on examination question performance between students in the traditional lecture section and the section using a flipped format.²⁰ Examination scores are often used as a proxy measure for learning; however, the ultimate goal is for students to be able to apply classroom learning to real-life situations, which is more difficult to assess.

FLIPPED CLASSROOM METHODOLOGIES

Numerous flipped classroom design possibilities exist, which makes it difficult to understand all the nuances of this approach. Kim et al identified nine crucial design principles for flipped classrooms (Appendix 1).²¹ Each of these principles follows a learner-centric teaching philosophy, which effectively incentivizes and guides students in their approach to learning. Some of the more salient principles are providing students an opportunity to gain first exposure to content prior to class, incentivizing them to prepare for class, and providing clear connection between in-class and out-of-class activities.

Similarly, Estes et al proposed a 3-stage, flipped classroom design framework consisting of preclass, in-class, and postclass learning activities.²² Although methods may vary, preclass activities are typically readings,¹¹ short recorded video or audio lectures, or some other form of computer-based instruction.^13,23 When properly designed, preclass work should not contain excessive detail, but should cover primary learning points.²⁴ Students should be provided guidance regarding preclass learning activities. One example of this guidance is to inform students what they need to be able to do or know in advance of class, rather than simply instructing them to read a specific chapter or watch a video.²⁵ Much of the success of the flipped approach depends on the interplay between preclass and in-class activities. Off-loaded preclass content must be presented in a fashion that students can readily comprehend and, be tied directly to in-class application.²⁶ If the two components are disconnected or if students can “succeed” without attending to off-loaded content then little may be gained from the change.

In-class activities, the second stage of the framework, may consist of a variety of methods for clarifying preclass activity concepts including: quizzes given at the beginning of class, group learning activities, problem solving, case discussions, or other active-learning methods that apply to different learning types.^17,27,28 Anecdotal discussions of flipped classrooms often focus on the off-loaded content stage, but the in-class activities are arguably the most important instructional component and should be given considerable forethought and attention. Classroom time is valuable and should make the best use of instructor knowledge, experience, and abilities to assist students in learning how to think like a pharmacist. The focus of classroom learning sessions should not be on the presentation of content, but on maintaining active student engagement with material.²⁹

Although not used in every situation, postclass learning activities in flipped classroom formats usually involve learning assessments and further application of skills or knowledge. Similar to traditionally-taught courses, class projects, portfolios, examinations, and other forms of authentic assessment are often used to determine and document achieved course competencies.²² These assessments are important not only to instructors, but for students who may need more feedback on their comprehension of material.

McLaughlin et al provided one example of a flipped classroom design in terms of off-loaded content and in-class activity for a basic pharmaceutics course. Course lectures were shortened, recorded, and made available online for students to view before class. In-class activities consisted of audience response and open questions, pair-and-share activities, student presentations, quizzes, and microlectures.¹³ As stated previously, the concepts behind flipped classrooms are not entirely new. Other teaching methodologies used in pharmacy education follow the same philosophy, and instructors can often adapt the design and active-learning techniques for a flipped classroom approach. Blended-learning methods are closely associated with flipped classrooms because they both shift educational activity to online settings and reduce classroom time for lecture or other types of learning tasks. Blended-learning methods have been used successfully for years in pharmacy courses^26,30,31 and offer examples of successful learning activities. Team-based learning (TBL) is another example of a specific technique used throughout pharmacy education.³² In TBL courses, students are responsible for preclass learning objectives, and class time is spent assessing knowledge and working in teams to complete a variety of activities.^19,33,34 Problem-based learning (PBL) is yet another example of a methodology in which classroom time is used for problem-solving in lieu of instructor lectures.³⁵

STUDENT PERSPECTIVES

Students accustomed to traditional didactic lectures may initially resist the concept of flipped classrooms because the onus of learning is shifted to them,³⁶ they perceive the workload will be too rigorous,²⁷ or they are anxious regarding potential classroom unsettledness.^1,24,37 An underlying fear of added workload and uncertainty of success is the basis for their concerns. These are legitimate fears rooted in years of familiarity with learning in traditional classroom environments and, therefore, may take considerable time to overcome. While early evidence suggests that this teaching method can be successful in promoting learning and other traits desired of students, it is not substantial enough to readily sway all students to this approach.

Students can have a different perception than faculty members regarding instructional strategies that require increased personal responsibility for learning. “Teaching ourselves” is a phrase commonly used negatively by students to describe instructional practices in which didactic lectures are limited and students must learn fundamental content on their own without considerable guidance from instructors.³⁸ This also leads students to believe they have “extra” work because they must complete all preclass activities to keep pace in the class. A third expressed fear among some students is that group discussions and problem-solving activities create an unsettled classroom (ie, a chaotic classroom environment in which students may feel lost).¹⁰

Even with those fears, research of flipped classroom methodologies in other disciplines such as business, engineering, medicine, and nursing indicate that students typically respond favorably to this type of instruction after experiencing it.^1,36,39-42 Most students who experience flipped courses prefer an inverted class format containing hands-on, problem-solving activities in class as opposed to a traditional lecture format.³⁸ Although research in pharmacy education is sparse, results from the few existing studies are consistent with those from other disciplines. In a pharmaceutics course delivered to 22 students on two satellite campus, 89.5% of students preferred the flipped format after completing the course compared to 34.6% before the course.¹³ Survey results of pharmacy student (N=64) perceptions of flipped learning within another pharmaceutics course indicated that students favored flipped learning over a traditional lecture design.⁴³ Similarly, pharmacy students (N=286) in a flipped nonprescription drugs/self-care course were surveyed regarding teaching and delivery styles. Following course completion, 30% (n=86) favored the flipped format, 48% (n=137) favored a combination of flipped and traditional formats, and 19% (n=54) favored traditional lecture format.²⁸

McLaughlin et al’s survey research of students (N=162) in a required first-year pharmaceutics course revealed significant differences between precourse and postcourse attitudes toward the flipped classroom learning environment. Preferences for the flipped classroom approach increased from 27.3% (n=41) prior to the course to 84.6% (n=126) after the course. Furthermore, attendance was higher in the flipped classroom (p<0.05) compared to the previous year.¹³ Pierce and Fox found that student perceptions regarding a flipped classroom renal pharmacotherapy module were also favorable, with 80% (n=42) of students indicating the module increased their efficacy for the final examination and 62% (n=32) indicating a desire for more instructors to use this form of teaching.⁴⁴

Wong et al’s study revealed that student perceptions of effectiveness of preclass lecture materials followed by in-class case-based exercises for teaching cardiac arrhythmias varied according to the topical area. Sixty-three percent (n=43) strongly agreed that those methods were more effective than traditional methods for learning therapeutics. However, only a minority of students were in favor of flipped classroom methods for the basic sciences (35%, n=24) and pharmacology (13%, n=9). One potential explanation for the negative perceptions was student concerns regarding the excessive length of the pharmacology recordings.¹⁸

CONSIDERATIONS AND QUESTIONS

Similar to other instructional approaches, a number of philosophical, technical, and logistical issues must be considered when implementing flipped classroom methodologies. Many educators advocate flipped classroom methods and extoll the virtues of how these methods can engage students in their own learning processes, add value to classroom time, and improve overall learning outcomes.^12,24,45-48 Initially, however, some faculty members are reluctant to adopt such strategies for a variety of reasons. Some are unsure of what they will do during class time and fear they will be unable to deliver the material adequately,⁴⁸ especially to large enrollment classes.⁴⁹ Some fear the increased time required to implement the method.^1,11 Others cite caution with regard to accepting the value of technology-enabled strategies.⁴⁸

Converting from a traditional classroom teaching environment to a flipped classroom environment is not always easy. Initial development of learning materials for a flipped classroom may require a significant time investment by faculty members.¹⁷ A flipped classroom design for a pharmaceutics course required 127% more faculty time to implement than in the previous year’s traditional lecture format, but that time was expected to decrease once the course was established.¹³ Moreover, after the initial offering of a redesigned self-care pharmacy course, time spent planning lectures returned to the same level as before the redesign year.¹¹

Like other forms of nonlecture-driven education, logistical issues should be considered before implementation. First, instructors should be cognizant of time requirements necessary for students to adequately complete out-of-class assignments. Faculty members must determine appropriate levels of students’ out-of-class work, especially with condensed courses (ie, those taught over a shorter time period such as a 4-week summer course) and/or an entire curricula of flipped courses. Communication among all instructors for a student cohort may be required to prevent unreasonable amounts of “homework.” Flexibility of access is one of the positive aspects of off-loading content, but students can be disadvantaged if instructors collectively assign too much content, do not provide adequate lead time, or put time restrictions on when assignments such as quizzes can be completed. Out-of-class assignments may need to be limited and/or fewer in-class sessions required for students to adequately prepare. For example, instructors for a redesigned first-year pharmaceutics course converted 29 hours of lecture into 14.4 hours of self-paced recorded videos that prepared students for the in-class learning activities. They deemed this reduction in lecture hours necessary to allow for ideal student preparation and to emphasize only the most critical concepts.¹³

Course enrollment sizes should also be factored into design considerations. Methods used in flipping smaller courses may not be as feasible with large class enrollments, however advocates maintain that flipped classrooms can still benefit all course sizes.^{13-15,24,36,50} It may be more difficult to design and facilitate effective in-class activities for larger classrooms, but instructors have numerous options. The use of group and team-based learning activities is the most common example, but there are a variety of other ways to interact with a large classroom of students.¹¹ Classroom discussions, the use of student response clickers, and case studies allow students to be individually accessible throughout physical class time.⁵⁰ Forms of digital technology, such as learning management systems, can also be used for individual assignments or in-class quizzes.⁵¹

Because formal flipped classroom methods are relatively new to higher education, research regarding their effectiveness has only recently become available in the literature. Although the literature informs educators of the benefits of using flipped classroom methodologies, several questions remain unanswered and deserve further study.

One question for programs that institute a curriculum-wide flipped classroom approach is determining if there are different/new characteristics that predict student success. Pharmacy College Admission Test (PCAT) scores and grade point averages (GPA) have long been considered the primary predictors of student success in pharmacy curricula.^52-54 However, those are primarily knowledge-based criteria and ones in which content memorization and test-taking skills can lead to success. Because classroom activities (and presumably assessments) in flipped classrooms require more problem-solving and application skills, content memorization and test-taking skills may be insufficient. Perhaps schools need to examine other nonquantitative student characteristics such as intellectual curiosity, personal responsibility, reasoning skills, etc., in order to identify and admit students most likely to succeed.

Another question that may be of concern to faculty members is if graduates of flipped classrooms will still possess understanding of fundamental pharmacy “facts.” The flipped classroom philosophy eschews spending class time on knowledge-based content in favor of higher-order thinking and problem solving. Theoretically, students will still need to learn that material through preclass assignments to perform well in class, but some fear that memorization of fundamental knowledge and facts will be lost with teaching methods that focus on critical thinking and problem solving. Research in this area is still too new to reveal any positive or negative long-term effects. A similar question is whether flipped classrooms improve the problem-solving and critical-thinking skills they are hypothesized to and how that will be assessed.

A final question that remains unanswered is whether there are strategies and techniques within flipped classrooms that are either easier to implement or are more effective than others. One of the difficulties that faculty members appear to have with course redesign is determining exactly what to choose from the seemingly infinite numbers of options for in-class activities. As more research is conducted, strategies need to be studied so that best practices can be developed with regard to off-loaded content, technologies, in-class activities, and assessment.

OVERCOMING RESISTENCE TO THE FLIPPED CLASSROOM

Although traditional lectures are removed from flipped classrooms, they are not without value. At the right time, for the right purpose, by the right instructor, lectures can be a valuable form of teaching, but they should not be the only or primary form of teaching in today's educational environment. It should also be recognized that many instructors have already abandoned the lecture in favor of more active-learning activities within the classroom. The advantages of flipped classrooms can be experienced through the addition of hands-on, problem-solving activities. Flipped classrooms are the next step in that evolution and as with most changes, there are both concerns and benefits.

While flipped classroom teaching methods offer several advantages to educators, haphazard implementation will almost surely result in disgruntled faculty members or students and a compromised learning environment. As the literature indicates, there are a variety of issues to consider when instituting a dramatic change in teaching approach. This paradigm change reflects a shift in what it means to be a teacher and a student. Although most reports indicate that faculty members and students appreciate the change after it has occurred, several years of ingrained habits and beliefs must be overcome. Both faculty members and students may need convincing that the change will be beneficial.

Students whose perception that learning only occurs while listening to an instructor lecture may initially complain that flipped classroom teaching methods equate to students teaching themselves while faculty members stand idly by.³⁸ There may even be some faculty members who reject the idea that students can learn without actually hearing/seeing the instructor tell them. A common student misperception about flipped classrooms is that students learn completely on their own without guidance and support from faculty members.³⁹ Both parties would have legitimate complaints if those perceptions were accurate. However, flipped classrooms do not involve less of a teaching commitment from instructors. They actually may require more teaching, interaction, and communication. That teaching may look and sound differently than what some have experienced, but the guidance and support required by instructors is crucial, and in reality, is probably closer to how faculty members prefer to teach and be taught.

Students may be more accountable for their own learning in flipped classrooms, but that does not necessarily mean an increased workload.¹ For some students, workload may increase but only to the extent that is already expected by instructors. Strategies for studying may need to change because “coming to class prepared to learn” will mean more than being punctual and ready to take notes. For some students, this format may better align with how they prefer to learn, which is to obtain and study basic content on their own and use class time for interacting with the instructor, the content, and their classmates.

Student opposition to an abrupt transition in classroom dynamics from traditional lecture to a flipped classroom is understandable. In essence, this is changing the rules for how one becomes successful in a course. Spending considerable amounts of time memorizing content, “cramming” for tests, and then forgetting much of it postexamination, may no longer work when reasoning and rationale becomes paramount. Time spent in the classroom is an opportunity for students to better understand the reasoning, rather than the means of receiving information. Although some students may initially resist these changes, it is incumbent upon pharmacy educators to move students toward self-directed learning that prepares students for their future beyond pharmacy school.⁵⁵

Changing to a flipped classroom paradigm may actually be more bothersome to faculty members than to students. Some faculty members may struggle with releasing the reliance on their role as content-deliverer, especially if they have been considered by themselves or others to be a “great teacher.” Those faculty members should be encouraged to focus on the value they bring to the classroom and to offer something that students cannot get by reading a book or watching a video. Those who are truly great at teaching will adapt and continue to be as good or perhaps even better teachers regardless of the teaching environment.

The transition to a flipped environment will involve additional work on behalf of faculty members.^13,17 The return on investment of faculty time is a big consideration. However, time investment should not be viewed merely as “extra work,” but as potential quality improvement measures expected of any professional. Like other forms of quality improvement, the increased work is realized early in the development phase, but theoretically returns to “normal levels” in subsequent years.

As with any new strategy, implementation and execution are important components of this teaching philosophy. Educational environments may vary widely with regard to class sizes, educational philosophies, human and technical resources, and student composition. These are all important elements of any curriculum and add to the complexity of evaluating educational methods. Not all flipped classrooms “look the same,” and the success of each depends upon appropriate instructional design to achieve desired outcomes. Providing faculty members with educational development opportunities on how to successfully utilize flipped classroom methods is vital,⁵⁶ as is providing sufficient time and technical and human resources to implement the approach.

The actual design of any flipped classroom is crucial and perhaps an area of trepidation for faculty members considering a redesign. Because most faculty members did not experience this type of teaching as a student, they may have difficulty comprehending what it should actually look like. Instructional designers or learning experts should be involved with topic, module, and course design in order to avoid pitfalls and realize the full potential of this methodology. Faculty members must avoid creating another form of instruction in which most students can “hide” and still be successful. One of the more important and perhaps controversial aspects of a flipped classroom paradigm is that faculty members must allow students to fail if they do not take personal responsibility for their learning. Faculty members should make efforts to reach students who are struggling or who seek guidance, but enabling students who do not take it upon themselves to comprehend foundational concepts is detrimental to their development as professionals.

Finally, assessment methods may also need to change. Teaching for application but testing primarily for knowledge will lead to dissonance and likely result in student dissatisfaction. As with other forms of teaching, students should be tested how they are taught. Students will always adjust their personal learning habits to the incentives of grades, and faculty members should ensure that as much as possible, grades reflect the knowledge, skills, and attitudes we desire. Existing research provides best practices on implementation of flipped classroom methodologies. Appendix 2 offers a series of summarized recommendations for faculty members considering a transition to flipped classroom teaching.

CONCLUSION

Benjamin Franklin once stated, “Tell me and I forget, teach me and I may remember, involve me and I learn.”⁵⁷ Franklin’s quote embodies the core of the flipped classroom concept. When properly implemented, flipped classroom environments are primarily a positive experience for both faculty members and student pharmacists who learn to appreciate the increased engagement. The ever-evolving landscape of health care requires a curriculum that enables students to think through problem situations and enter the workforce confident in their ability to provide high-quality patient-centered care. Medical educators have even recommended that the flipped classroom philosophy form the foundation of a new model of medical education. Biomedical information is advancing faster than can be taught and students possess the digital skills to learn outside the classroom, therefore the proposed model should be one in which only foundational content be covered, while the rest of instruction concentrates on developing deeper intellectual and medical thinking skills necessary to operate in this new age.⁴¹

Rapid advancements in technology have made flipped classrooms possible by making fundamental facts and knowledge available for student review before actual class time. In doing so, class time can be focused on applying conceptual material in real-life clinical settings and learning how to “think like a pharmacist.” Flipped classrooms may be initially met with resistance by students and faculty members alike, but the opportunities to improve teaching and learning practices should not be ignored. Students deserve to be taught in a way that best prepares them to practice within a highly demanding work environment. The principles behind flipped classrooms are designed to accomplish that.

Published research regarding flipped classrooms in pharmacy education is increasing, but much more is needed to substantiate this method as a preferred way of teaching and learning. In particular, educators need access to research-proven methods that can be duplicated in other courses. The almost limitless design possibilities may seem daunting to faculty members considering this method of teaching, and the academy would benefit from a white paper that provides best practices and detailed implementation guidance.

Appendix 1. Nine Design Principles for the Flipped Classroom

• 1. Provide an opportunity for students to gain first exposure to content prior to class

• 2. Provide an incentive for students to prepare for class

• 3. Provide a mechanism to assess student understanding

• 4. Provide clear connections between in-class and out-of-class activities

• 5. Provide clearly defined and well-structured guidance

• 6. Provide enough time for students to complete the assignments

• 7. Provide facilitation for building a learning community

• 8. Provide prompt/adaptive feedback on individual or group works

• 9. Provide technologies familiar and easy to access

Appendix 2. Recommendations for Implementing a Flipped Classroom

• 1. Use instructional designers to determine what content should be offloaded and how in-class learning activities should be designed

• 2. Provide numerous individual and group-based faculty development opportunities to assist faculty members in the paradigm change

• 3. Recognize that the increased nature of classroom dynamics may result in the need for additional personnel (eg, faculty members, graduate assistants, residents) in the classroom

• 4. Recognize that “changing the rules” will cause angst among many students who may themselves need to be re-educated on how one is successful in flipped classrooms

• 5. Link all preclass and in-class activities so it is necessary for students to complete both to be successful

• 6. Provide guidance to students so they understand exactly what they need to know or be able to do when they come to class

• 7. Provide an opportunity for students to ask questions and/or clarify information contained in off-loaded course materials

• 8. Focus in-class activities on ways that you can add value in helping students think like experts

• 9. Provide formative assessment opportunities for students to understand what they do and do not know

• 10. Resist the urge to “reteach” content to students who do not prepare accordingly. Be prepared to let them fail in order for them to comprehend what it means to be accountable as a professional

• 11. Be careful not to burden students with too much off-loaded content

• 12. Assess knowledge, skills, behaviors, and attitudes that you desire from students. Do not teach critical thinking and problem solving, but test only for knowledge

• 13. Be prepared to adjust teaching and learning activities as you discover what contributed to learning and what did not