The Future of Medical Education: Assessing the Impact of Interventions on Long-Term Retention and Clinical Care

The study reported by Dolan et al¹ in this issue of the Journal of Graduate Medical Education epitomizes the type of research that is sorely needed in medical education. The authors conducted a randomized controlled trial in which they assigned internal medicine residents at a large academic practice to receive either the standard curriculum in fracture prevention, or a standard curriculum plus repeated, spaced practice on the material over a 3- to 6-month period. They found that the intervention with repeated practice produced better retention of knowledge 10 months later, and improved the quality of clinical care (bone density screening rates, appropriate use of bisphosphonates, but not FRAX score reporting) provided by the residents during the study. This study is innovative and important in that it assesses the long-term consequences of an educational intervention, and measures its impact on both knowledge and clinical practice.

In education at all levels and in all fields, we tend to focus on short-term outcomes. That is, learning is generally assessed during, or immediately after, the time it occurs (eg, questions posed to students during a lecture, quizzes at the end of class, or final examination after a course). Short-term outcomes are important for many reasons, such as formative and summative assessment. However, when we focus exclusively on short-term outcomes, we make an assumption that is often false: we assume that short-term performance is a good predictor of performance over longer periods of time. Unfortunately, mastery demonstrated during or immediately after learning can be easily lost in the following weeks and months without continued practice.^2,3 For example, numerous studies have found that a substantial portion of the basic knowledge and skills acquired in medical school is forgotten by the time individuals enter graduate training, let alone practice.^4–7 Similar patterns occur in graduate medical education, with residents forgetting knowledge and skills that are fundamental to their training.^8–12

One conclusion that can be drawn from studies that assess long-term outcomes is that we can be shortsighted in how we approach educational interventions. We often devote a substantial amount of effort to facilitating initial learning, but comparatively little effort to maintaining it. For example, a recent study¹³ assessed the short-term and long-term consequences of redesigning a lecture-based, preclinical pediatrics course to incorporate team-based learning, a pedagogical approach that promotes active learning. Through comparing a group that received the lecture-based version of the course to a group that received the team-based learning version, the study tracked student knowledge of core concepts from the course over time. When knowledge was measured after the end of course, the students who had taken the team-based learning version performed substantially better than students in the lecture-based version. However, when students were given a follow-up knowledge assessment prior to their clerkship, the learning gains in the team-based learning group had disappeared, and the 2 groups performed at the same level. Such findings demonstrate how devoting substantial effort to facilitating initial learning can yield benefits, but those benefits can be easily lost without efforts to maintain knowledge and skills afterward.

Given that learning gains can be easily lost in the absence of continued practice, how can we help medical students, residents, and other health professionals maintain the knowledge and skills that they acquire during training? The intervention implemented by Dolan and colleagues¹ provides a possible template. The intervention incorporates several mechanisms known to promote long-term retention and deeper understanding: retrieval practice, feedback, and spaced repetitions.^14,15 Retrieval practice refers to the act of retrieving information from memory (eg, solving a practice problem or answering a test question), which is a potent learning event.^16,17 Providing feedback increases the benefits of retrieval practice by correcting errors,¹⁸ maintaining correct responses,¹⁹ and increasing understanding.²⁰ When such practice is spaced or distributed over time, it is more effective than massed practice (eg, cramming).^21,22 Numerous studies have shown the efficacy of interventions that incorporate these 3 mechanisms on the retention and transfer of knowledge in undergraduate, graduate, and continuing medical education.^23–26 To our knowledge, however, the research reported by Dolan and colleagues¹ is the first that links such an intervention to improved quality of clinical care.

Of course, it is challenging to assess long-term outcomes and determine the amount of continued practice needed to maintain knowledge and skills that are not regularly used in clinical practice. However, here too Dolan and colleagues¹ present a potential solution: the use of technology. Advances in technology are providing educators with powerful new tools that are rapidly changing how people learn both inside and outside the classroom. At a minimum, technology can make providing continued practice more flexible, efficient, inexpensive, and scalable through automation. Yet, the real promise lies in adaptive technology that provides a personalized experience for each learner because it has the potential to exponentially increase the effectiveness and impact of educational interventions.²⁷ Technology is also transforming the way in which health records are kept, providing new opportunities to link learning in the classroom to practice in clinical settings. The ability to assess the impact of educational interventions on clinical care is critical to improving medical education.

In conclusion, we think that the innovative approach taken by Dolan and colleagues¹ represents the future of medical education. The training required to become a health professional is substantial. In order to ensure that the learning that occurs during each phase of training is retained over time, more effort must be devoted to maintaining the knowledge and skills acquired. We must also assess long-term outcomes, including the quality of clinical care, so that we can accurately judge the effectiveness of educational interventions. With medical interventions, health professionals routinely follow up with their patients: a surgery is never assumed to be a success immediately after the operation, and a drug is never presumed to be effective based solely on the initial administration. The same approach should be applied to interventions in medical education.