Effectiveness of a modified Continuum curriculum for medical students

Abstract

Background:

Continuum: Lifelong Learning in Neurology® is a well-regarded and widely used continuing medical education tool published by the American Academy of Neurology. The objective of this study was to test the effectiveness of a modified version of the Dementia module of the Continuum curriculum, adapted for medical students rotating on their neurology clerkship, in increasing medical knowledge of dementia.

Methods:

A multisite longitudinal randomized controlled design was used. Medical students rotating on their Neurology clerkship were recruited from 2 US medical schools. Participants completed 10 multiple-choice questions, 1 fill-in-the-blank, and 1 patient case simulation question to assess medical knowledge of the most prevalent dementias pre- and post-curriculum implementation. All students received their standard dementia curriculum (45-minute live didactic presentation on dementia along with a copy of the slides in handout form). Students were randomized to either the intervention (standard + Continuum curriculum) or control (standard curriculum alone) group. Data collection and outcomes assessment was optimized via an interactive audience response system (pretest) and Web-based survey/database tool (post-test and student satisfaction surveys).

Results:

From pre- to post-clinical clerkship, medical students completing the Continuum dementia curriculum in addition to standard clerkship curriculum demonstrated significantly greater increases in medical knowledge of dementia, relative to students completing only the standard curriculum. Subscores were significantly higher among Continuum-trained students on questions regarding Alzheimer disease (AD), frontotemporal lobar dementia, Lewy body dementia, AD treatment fill-in-the-blank, and AD patient case simulation.

Conclusions:

The Continuum: Dementia for Medical Students curriculum provided an inexpensive and readily implementable means for improving medical knowledge of dementia. Improved performance on an AD patient case simulation may be considered a surrogate marker for optimized patient care.

Continuum: Lifelong Learning in Neurology® is the American Academy of Neurology's (AAN) self-study continuing medical education publication. Continuum is published 6 times per year and issues include a combination of diagnostic and treatment outlines, clinical case studies, a topic-relevant ethics case, detailed patient management problem, and multiple-choice self-assessment questions. The American Board of Psychiatry and Neurology has approved Continuum as part of a comprehensive lifelong learning program.

While Continuum has been developed specifically for the practicing clinician, medical students may also have much to gain from incorporating this structured, evidence-based resource into their educational armamentarium. Under the guidance of the Undergraduate Education Subcommittee (UES) of the AAN, a condensed medical student version of Continuum: Dementia was developed. The impetus for this project was a general UES consensus that there was a need for a more standardized, effectiveness-proven medical student neurology curriculum (as mandated by the Liaison Council for Medical Education, revised ED-1-A). The topic of dementia was then selected as it was the next sequential topic to be published in the Continuum series.

A pilot study was first performed, which suggested that reading the condensed version of Continuum led to increases in medical knowledge. Pilot study data were collected on a group, rather than individual, level. The nonrandomized, cohort pilot study of this module among medical students from 2 institutions demonstrated that significant increases were seen in 7 of 11 of the test items. In addition, the Continuum curriculum was favorably rated by students on a satisfaction scale related to the perceived utility of the Continuum curriculum in improving patient care.¹ These data suggested that further study was warranted to assess the effectiveness of the Continuum curriculum as a teaching tool for students. We hypothesized that adding the modified Continuum curriculum to the standard neurology clerkship Dementia curriculum will lead to improvement in medical knowledge about dementia.

METHODS

Curriculum development.

In conjunction with the AAN UES, a working group was formed in October 2006 to discuss and develop a prototype of Continuum for medical students in order to provide neurology clerkship directors additional resources to enhance education. Over subsequent months and through collaborative efforts between the Publications Senior Manager and Managing Editor of Continuum, a core of 11 senior medical student reviewers from 2 medical schools, a member of the UES working group specializing in Alzheimer disease (AD) and other related neurodegenerative dementias, and the Chair of UES, a condensed and edited version of Continuum: Dementia was developed. Using a practice pattern–based model for curriculum development,^2–4 a 30-page comprehensive review on dementia topics including AD, mild cognitive impairment (MCI), dementia with Lewy bodies (DLB), frontotemporal lobar degeneration (FTLD), vascular dementia (VaD), and normal pressure hydrocephalus (NPH) was finalized in January 2007. Core content for AD (18 pages) was abbreviated with the following subheadings: Summary (1 paragraph), Disease Overview, Epidemiology, Neurophysiology, Neuropathology, Diagnostic Criteria, Clinical Evaluation, Case Presentation, Neurologic Examination, Laboratory Studies, and Treatment. The only substantive changes made to this text were in the treatment section, updating the most recent Food and Drug Administration–approved formulations of acetylcholinesterase inhibitor medications. Core content for the remaining chapters (12 pages) was abbreviated by including a 1-paragraph summary, followed by a listing of Key Points that were identical to those in Continuum: Dementia.

Study design.

This single-blind randomized control trial incorporated a 2 × 2 mixed factorial design. After informed consent and pretesting, all participants were randomized to either intervention (Continuum curriculum) or control (standard clerkship training without Continuum curriculum) groups. Therefore, time (pre- vs post-clerkship) was the within-subjects factor and group (experimental vs control) was the between-subjects factor.

Participants.

During the first week of rotating on their neurology clerkship, all third- and fourth-year medical students at the University of Miami Miller School of Medicine and the Joan and Sanford I. Weill Medical College of Cornell University in 2008 were offered the opportunity to participate. A total of 226 medical students were enrolled, approximately during the first day of beginning their neurology clerkship. A total of 128 students were enrolled at the University of Miami, and 98 students were enrolled at Cornell University. Seventeen students (∼7%) declined participation, the majority of whom (n = 16) were fourth-year students (figure 1). No compensation was provided for participating in the study.

Figure 1. Enrollment and randomization flow chart.

Standard protocol approvals, registrations, and patient consents.

The study received approval from the Institutional Review Boards of University of Miami and Cornell University. All students signed an informed consent. Students were informed that participation or nonparticipation in the study would have no effect on their clerkship grade or course evaluation.

Procedure.

After enrollment, students were administered a pretest. Preintervention testing consisted of 10 multiple-choice questions (MCQ) (performed via an interactive audience response system; Turning Technologies), a fill-in-the-blank question, and a patient case simulation. To avoid student sharing of questions or answers, the audience response system displayed each question visually via PowerPoint, and students were not allowed to take notes during the pretest. After taking the pretest, the students were then randomized to 1 of 2 groups (intervention vs control) using a random number generator. The intervention group received the standard clerkship dementia curriculum (live didactic slide presentation on dementia and a copy of the slides as handouts) plus the Continuum curriculum. The control group received the standard clerkship curriculum without the Continuum curriculum. To avoid cross-contamination, students in the intervention group were asked not to share their curricula with other students. Post-testing was collected via a Web-based survey database tool (surveymonkey.com) that relies on e-mail communication. Students received an e-mail–based post-test 1 month after clerkship completion. The e-mail directed students to a secure Web site that allowed them to answer MCQ, fill-in-the-blank, patient case simulation, and survey questions.

Primary outcome measure.

The primary outcome measure of the study is change in medical knowledge of the most prevalent dementias as assessed by total MCQ and fill-in-the-blank scores from pre- to post-clerkship.

The 10 MCQ were derived from self-study questions that were published with the original Continuum: Dementia monograph.⁵ These questions were reviewed and edited by 2 neurology faculty with subspecialty training in dementia and related cognitive disorders. These questions were initially used in the pilot study that preceded this trial. All test question content was aligned with the standard neurology clerkship curricula at each institution. This content was covered in either the slide presentation on dementia moderated by the clerkship directors or in the dementia handout materials given to students in the clerkship syllabus. The pilot study used these questions (Cognitive Impairment Assessment Questionnaire [CIAQ]) which were answered and later reviewed by 37 third-year medical students. Comprehensive post-implementation surveys were collected which assessed student level of agreement with a variety of question quality metrics.¹ Question selection was aligned with practice pattern data and included the most salient medical knowledge to best assess practical knowledge of assessing and treating cognitively impaired individuals. Evidence-based diagnostic and therapeutic learning points were assessed via 4 AD, 1 MCI, 1 DLB, 1 FTLD, 1 VaD, and 1 NPH question.

Secondary outcome measures.

Secondary outcome measures assessed the effect of Continuum on individual topic scores (AD, MCI, FTLD, DLB, VaD, NPH) and on a patient case simulation question (AD vs age-related memory loss with pseudodementia of depression).

Statistical analysis.

In order to determine whether the change in scores on the MCQ depended upon whether students received the Continuum curriculum, a repeated-measures analysis of variance (ANOVA) was used to test the interaction between time (pre- to post-clerkship) and intervention (Continuum curriculum vs control). Analyses were repeated controlling for site. In addition, repeated-measures ANOVAs were conducted by individual site.

In addition to analysis of pooled data from both institutions, analysis was also stratified by institution (intervention group vs control group at Cornell; intervention group vs control group at Miami). Reasons for stratification included differences in student background and lack of preexisting dementia curriculum alignment between sites. Stratification was also performed to control for interinstitutional differences between medical students. All data were stored in a secure, Web-based database program, with export capabilities to Excel and SPSS. Data analysis was assisted by a biostatistician (C.O.).

RESULTS

Baseline scores did not differ by site (NS, p = 0.11) or group (NS, p = 0.49) (table 1). There was a time × intervention interaction, such that increases in CIAQ scores depended upon completion of the Continuum curriculum, F_1,224 = 115, p < 0.0005. Students who received, vs did not receive, the Continuum curriculum scored higher on post-testing (67.97% ± 14.17 vs 42.78% ± 10.78) and had greater score increases (25.19 ± 14.36 vs. 5.74 ± 12.87) from pre- to post-testing (both p < 0.0005) (figure 2). Controlling for site had no effect on results and results repeated by site were nearly identical to combined results (table 1).

Table 1.

Mean multiple-choice question scores by site and time point

^aStatistic from repeated-measures analysis of variance.

Figure 2. Pre- and post- Continuum intervention test scores between groups.

Although increases were seen in both groups (p < 0.0005), the increase in the group receiving Continuum showed significantly greater increases in multiple-choice question scores (25.2 ± 14.4) as compared to the group not receiving Continuum (5.7 ± 12.9).

Among students receiving the Continuum curriculum, the greatest subtopic score improvements (averaged across sites) were FTLD (22% pre vs 72% post), DLB (30% vs 75%), AD patient case simulation (28% vs 72%), and AD treatment fill-in-the-blank (21% vs 55%). All changes p < 0.0005.

In addition, the Continuum curriculum was favorably rated by students on a satisfaction scale related to the perceived utility of the curriculum in improving patient care, appropriateness of the curriculum for students, and worthy in terms of time invested in reading (table 2). The average time to read the entire monograph was 47 minutes.

Table 2.

Ratings of perceived utility of Continuum curriculum for improving patient care

DISCUSSION

The AAN defines education research as the “qualitative and quantitative study of hypothesis-driven observations or interventions on the acquisition of knowledge or training.”⁶ Little data exist in the literature studying the effectiveness of neurology curricula.⁷ In fact, most published data regarding medical student education in neurology are observational or survey-based.^8–10 A small study demonstrated that AAN practice parameters in Parkinson disease improved evidence-based practice.¹¹ In a recent survey of neurology clerkship directors, the majority report that they generate in-house syllabi and also recommend one of the various available textbooks.¹² No outcomes-based data exist to determine the effectiveness of any of these study tools in the acquisition of knowledge.

To our knowledge, this article presents the first multicenter, randomized control trial to study the effect of a new curricular tool in neurology medical student education. Based on the findings of our study, we can conclude that adding Continuum for Medical Students: Dementia to the standard curriculum improves medical knowledge over the standard curriculum alone. We cannot say that the Continuum curriculum should replace the standard curriculum because the study was not designed with a group that only utilized the Continuum curriculum. The authors did not feel that it would be ethical to withhold the standard curriculum and provide the students with an experimental curriculum alone since it could possibly adversely affect the final grade for the course.

A significant strength of this study is the multicenter nature of the design. This facilitates generalization of the results to other medical schools since the study sites, like most neurology clerkships, did not utilize one standard curriculum. Instead, the standard curriculum for each of the sites in the study was locally developed at each school. In addition, background work was done to pilot the curriculum and assess the MCQ as ecologically valid in relation to the frequency of specific dementia subtypes. Limitations of the study include restricted number of subtopic MCQ that limits interpretation of results. In addition, there was no validated measure of medical knowledge of the most prevalent dementias available for use in this study, necessitating the creation of a new measure. While MCQ were selected from those initially published in Continuum: Dementia, the answers to all test questions were provided to both groups, albeit in different forms. What differed between groups was the delivery method. The didactic presentation and slide handouts that were provided to both groups were curricularly aligned with and contained answers to all questions on the pre and post tests. We asked students to not share the curriculum with other students in the trial. To counter this possible contamination, we offered the curriculum to all students at the end of the study period. We also aimed to decrease cross-contamination by administering the MCQ via an electronic resource where each student had an individual response device and prohibited students from taking notes during the pretest examination.

While it is difficult to ascertain how improved performance on MCQ would truly apply to real-time behavior in the clinical setting, improved MCQ performance may be a surrogate marker for optimized patient care. Future studies should make an effort to study real-life patient outcomes, as mandated by the Liaison Committee on Medical Education and Accreditation Council for Graduate Medical Education. While this should be the goal, obstacles are numerous and the need to access protected health information adds a significant layer of complexity to such studies. In an effort to study surrogate markers of patient outcomes, we used a patient case simulation question where the correct patient diagnosis was age-related memory loss with pseudodementia of depression. Each incorrect management answer can be projected to have different monetary costs, such as starting such a patient unnecessarily on a cholinesterase inhibitor medication (∼$1,920/year) or NMDA antagonist medication (∼$2,760/year). These treatments may increase morbidity due to the potential for medication side effects. Other measures to assess utilization included response choices that offered an unnecessary neuroimaging study (∼$1,600) or genetic test (∼$400). These surrogate markers can then be tracked to assess an estimate of monetary costs and morbidity saved from preintervention to postintervention.¹³

Our study suggests that a medical student condensed version of Continuum: Dementia is an effective curricular tool in addition to the standard curriculum. The authors speculate a variety of reasons why this was superior to the standard approach. Given the sometimes overwhelming amount of reading expected of medical students, it is possible that the condensed Continuum curriculum provided a vehicle through which students can obtain vital information on dementia without being overloaded with extraneous detail. In addition, the chapter was updated with the latest diagnostic and treatment methods that otherwise may lag behind in printed book chapters. The cover page of the curriculum was printed with the Continuum logo and included a summary of the pilot study results, thus potentially improving buy-in. Several students commented in the free text portion of the survey that they were intrigued that the curriculum had been previously shown to increase medical knowledge specifically in students.

Given the relative brevity and low cost of this curriculum addition, it could be easily added to the standard medical student curriculum on a nationwide or global scale. As a result of the UES interest in a Continuum prototype for teaching medical students, the AAN now offers free online access to the Continuum series as a free benefit to all medical student members. The findings in this study suggest that similar modification of evidence-based continuing medical education publications, such as Continuum, should also be considered for future trials.

AUTHOR CONTRIBUTIONS

Statistical analysis was conducted by Dr. Christopher N. Ochner.

DISCLOSURE

Dr. Isaacson is supported by the National Institutes of Health Clinical Research LRP and serves on speakers' bureau for and has received speaker honoraria from Novartis and Accera, Inc. Dr. Ochner and Dr. Safdieh report no disclosures.