Introduction to Curriculum Development and Medical Education Scholarship for Resident Trainees: A Webinar Series

Abstract

Introduction

A common career aspiration among residents is to become a clinician-educator, though standard postgraduate training may not prepare trainees for the academic and scholarly requirements of this career. To address this need, we designed and implemented an asynchronous, interactive webinar series detailing a systematic approach to medical education research and scholarship. The series was piloted as part of a new track at the University of Chicago for residents interested in additional training and completing an educational learning project in medical education.

Methods

We aimed to use this series to introduce relevant frameworks in curriculum development, program evaluation, and learning theory. Materials associated with this publication include six webinars and corresponding summary reference handouts, discussion assignments, and answer keys. Additional materials include a faculty course director packet and sample feedback for discussion assignments. Each webinar is an 8- to 20-minute narrated presentation with goals and objectives, an overview of each session's content, and example vignettes. Residents viewed presentations and completed a two-part discussion assignment for each webinar, which included reflection on the educational material and vignettes, faculty feedback on this reflection, and application of webinar material and faculty feedback to their own experiences in medical education.

Results

All residents in the pilot completed the webinars and assignments. Residents’ reactions to the webinar series have been positive, and residents have commented that self-paced learning with directed faculty feedback is a desirable instructional method for this material.

Discussion

This series is well suited to introduce fundamental concepts in medical education scholarship and inspire self-directed study for motivated learners.

Educational Objectives

Webinar Series Objectives

By the end of the webinar series, learners will be able to:

• 1.Identify and describe the six steps of curriculum development: problem identification and general needs assessment, needs assessment of targeted learners, goals and objectives, educational strategies, implementation, and evaluation and feedback.
• 2.Develop and appraise elements of an educational program or curriculum using the following frameworks for evaluation and scholarship in medical education: Miller's pyramid for clinical competence, the Kirkpatrick model of program evaluation, and Glassick, et al.'s criteria for scholarship.
• 3.Describe principles of adult learning theory and compare and contrast different approaches, specifically, Kolb's experiential learning cycle and learning styles and VARK learning preferences.
• 4.Critically apply skills in curriculum design, development, and evaluation to their own experiences and educational learning projects in medical education scholarship.

Webinar-Specific Goals and Objectives

By the end of Webinar 1, learners will be able to:

• 1.Identify a problem for a curriculum to address.
• 2.Contrast general needs assessment and targeted needs assessment.
• 3.Name the FINER criteria for evaluating a scholarly question.
• 4.Describe Miller's pyramid for clinical competence and the Kirkpatrick model of program evaluation as frameworks for evaluation in medical education.

By the end of Webinar 2, learners will be able to:

• 1.Define a goal and an objective and contrast the two.
• 2.Identify the four elements of a specific measurable objective.
• 3.Name a goal for a specified curriculum and write specific measurable objectives for that specified curriculum.

By the end of Webinar 3, learners will be able to:

• 1.Describe Kolb's four-stage learning cycle.
• 2.Describe Kolb's different learning styles and how each corresponds to different instructional strategies.
• 3.Choose an appropriate instructional strategy for each step of Kolb's learning cycle for a curriculum.

By the end of Webinar 4, learners will be able to:

• 1.Describe basic principles of adult learning theory.
• 2.Classify their individual curricular objectives into the cognitive, affective, and psychomotor categories.
• 3.Choose an appropriate educational strategy for each curricular objective.

By the end of Webinar 5, learners will be able to:

• 1.Identify necessary personnel, time, facilities, and funding for their respective curricula.
• 2.Describe two potential barriers and corresponding solutions for implementation of a curriculum.
• 3.Design a schedule for piloting of their curriculum.

By the end of Webinar 6, learners will be able to:

• 1.Define and contrast assessment and evaluation.
• 2.Name Glassick et al.'s criteria for scholarship.
• 3.Describe common evaluation study designs and evaluation methods.
• 4.Describe common formats for dissemination and scholarship in medical education.

Introduction

Pursuing a career as a clinician-educator is a common aspiration and interest among trainees in graduate medical education (GME). Some factors associated with this goal include training at a research-oriented program as well as strong clinician-educator mentorship and role-modeling.¹ Unfortunately, it has also been demonstrated that trainee interest in clinician-educator careers tends to wane throughout residency training and that residents may be poorly prepared to enter a career as a clinician-educator.^1–3 Residents may gain teaching experience during their clinical service and are often supported in these efforts through residents-as-teachers programs.⁴ However, these programs focus primarily on clinical and bedside teaching skills and may lack any discussion of scholarship in medical education, a recognized and key component of a career as a clinician-educator.³

Curriculum development and evaluation are common avenues by which residents may gain access to scholarly projects in medical education, either by working on existing initiatives or by proposing new projects or curricula within their own residency programs or medical schools. Though these projects are often undertaken informally by residents, numerous frameworks exist to help guide and instruct on a scholarly approach to curriculum development, adult learning theory, and program evaluation.⁴ However, GME learners may not have the time, skills, or resources to utilize such frameworks in their own scholarly projects and may lack understanding of how to systematically plan, implement, and evaluate a curriculum.

A structured introduction to scholarship in medical education is one strategy to address the lack of training in these skills during postgraduate training and to continue to foster resident interest in careers as clinician-educators. We aimed to design and implement a curriculum that would provide such an introduction in the form of a webinar series, taking into account the time limitations of GME learners participating in clinical residency programs and the learning preferences of the current generation of trainees.^5,6

We designed this webinar series to serve as an introduction for GME learners to fundamental concepts in medical education research and scholarship, with a focus on curriculum development, evaluation, and adult learning theory. Our curriculum was implemented at the University of Chicago in 2014 and was piloted as part of a new GME track for residents interested in pursuing additional training and completing a mentored scholarly educational learning project (ELP) in medical education. The series aimed to provide an introduction to several well-known and commonly used major frameworks in medical education.^7–9

Methods

This webinar series consists of six parts. Each part includes a narrated webinar presentation (Appendices A–F); a summary handout of key terms, concepts, and suggestions for further readings (Appendices G–L); and a discussion packet (Appendices M–R) that includes a two-part discussion and reflection exercise with a corresponding answer key. Part 1 of the discussion exercise involves reflection on hypothetical vignettes, and Part 2 of the discussion exercise stimulates learners to apply each webinar's content to their own experience in medical education and ELP, guided by faculty feedback on Part 1. A course director packet is included (Appendix S), along with sample feedback that was provided for resident answers to the discussion exercises (Appendices T–Y).

The series is intended for residents in any discipline with an interest in medical education. Webinars and discussion exercises were designed as an asynchronous exercise for residents to complete at their own pace given the variety of specialties and schedules represented among learners. We selected an asynchronous format for our curriculum to maximize delivery of educational content while respecting the busy time demands of GME learners in clinical training programs. Additionally, we wanted to create content that respected and aligned with learning preferences of millennial learners, the generation most widely represented by current residency trainees.⁶ In order to do this, we constructed the series to allow residents to complete webinars at their own pace but brought learners and faculty facilitators back together with accompanying assignments that fostered interaction, feedback, and application.

At our program, the curriculum has been implemented largely during the second and third years of postgraduate clinical training. As noted, the webinar series was piloted as part of a larger residency track for residents interested in medical education, and these residents hailed from multiple specialties with highly variable clinical schedules, call responsibilities, and training environments (e.g., ward-based or ambulatory schedules for residents in internal medicine and pediatric residencies, shift scheduling for residents in emergency medicine residencies, and operating room responsibilities for residents in surgical and anesthesia residencies).

Target Audience

The target audience is learners in residency training with the following prerequisites:

• •Interest in pursuing additional training in medical education and scholarship.
• •Basic knowledge of terminology of different components of a curriculum (e.g., goals, objectives, educational strategies).
• •Exposure to an ELP in medical education—a project related to curriculum development in which the learner is directly participating or sufficiently involved in the process to reflect upon webinar content in discussion exercises.

Logistics

The webinar series and corresponding discussion documents were housed on an online learning management system (Blackboard Learn Suite). Residents were given access to the online classroom, and a schedule with due dates was provided for the entirety of the academic year, as participating residents were involved in a medical education track that spanned the academic year. Residents were instructed to view the webinars at their own convenience, then complete the assigned discussion sheet and email it back to the course directors by the designated due date. Course directors were University of Chicago faculty members who were clinician-educators with additional training and experience in curriculum development and program evaluation. In the pilot, the course directors were directly involved in development of the webinar series and frequently served as the ELP mentors for participants as well. The webinar series was piloted in the 2014–2015 academic year and underwent its second implementation with the second class of track participants in the 2015–2016 academic year.

The discussion exercises were designed such that Part 1 of the assignment asks the GME learner to apply the webinar material to an example vignette that is introduced in each webinar. The resident views the webinar and completes Part 1, then emails the completed Part 1 document to the course director. The course director returns an annotated Part 1 discussion sheet with specific feedback on the resident's answers to the questions, along with the Part 1 answer key. The resident then moves on to Part 2 of the discussion exercise, which prompts reflection on questions that apply the webinar's content specifically to the resident's own experience in medical education and ELP. The resident then emails the completed Part 2 document to the ELP faculty mentor, who reviews it and again returns an annotated Part 2 discussion sheet with additional feedback on the responses.

Originally, the webinars were designed as a one-part exercise, but we received feedback from the GME learners and faculty members after the first set of webinar assignments were completed that the material might be better suited for a two-part format. Additionally, it was noted that providing an answer key might better enhance understanding. In response to this feedback, we created the two-set discussion exercise process as detailed above in order to ensure that residents receive faculty feedback to help facilitate a better understanding and application of the webinar material to their own experience and ELPs.

Preparation

Course directors should view the six webinars, which range from 8 to 20 minutes in duration, and review the source material if applicable. They should review the course director packet as well as the discussion packets (which include the discussion sheets and answer keys) to familiarize themselves with the assignment that the residents will be submitting. Though the asynchronous nature of the webinar series does not necessarily require a deadline, course directors may find it useful to provide a time course or schedule for expected completion of each series by residents, as well as a reasonable expectation for faculty feedback to be returned to the participants. For example, for the second year of the webinar series (academic year 2015–2016), our timetable was set as follows:

November 2015—Webinars 1 and 2

• •November 2: Part 1 discussion sheet due to course director via email.
• •November 9: Part 1 discussion sheet returned to resident with annotated feedback and Part 1 answer key via email.
• •November 16: Part 2 discussion sheet due to faculty ELP mentor via email.
• •November 23: Part 2 discussion sheet returned to resident with annotated feedback via email.

March-April 2016—Webinars 3 and 4

• •March 14: Part 1 discussion sheet due to course director via email.
• •March 21: Part 1 discussion sheet returned to resident with annotated feedback and Part 1 answer key via email.
• •March 28: Part 2 discussion sheet due to faculty ELP mentor via email.
• •April 1: Part 2 discussion sheet returned to resident with annotated feedback via email.

May 2016—Webinars 5 and 6

• •May 2: Part 1 discussion sheet due to course director via email.
• •May 9: Part 1 discussion sheet returned to resident with annotated feedback and Part 1 answer key via email.
• •May 16: Part 2 discussion sheet due to faculty ELP mentor via email.
• •May 20: Part 2 discussion sheet returned to resident with annotated feedback via email.

We found it useful to house the webinars and discussion packet materials on an online classroom that allowed for distribution and tracking views by participating GME learners. However, the asynchronous nature of the curriculum makes it such that these files could also be distributed via email at the appropriate intervals.

Results

Twenty-two residents from nine different specialties have participated in our curriculum to date, 11 in each class. All participants completed 100% of the webinars. The majority of residents entering into the program were PGY2 (13 out of 22, 59%) and PGY3 residents (six out of 22, 27%); the remaining three residents were advanced PGY4 and above residents in surgical subspecialties participating in the program during a research or elective year.

At baseline, most of our learners (20 out of 22) had received some type of medical education training in the past, with the majority (11 out of 20) receiving lecture-based training. About half (11 out of 20) had reported experience with prior medical education scholarship, including curriculum development, abstract preparation, writing a book chapter, or participating in social media (e.g., Twitter). Only a minority (two out of 20) of our learners had received web-based training prior to our curriculum. After completion of our webinar series, 86% (19 out of 22) of respondents were satisfied or very satisfied with the quality of the webinars. Furthermore, 86% of participants from the first class agreed or strongly agreed that they would recommend participation to future classes.

Comments for the webinar series were generally positive:

• •“I really liked the webinar series as it was much easier to complete on my own time.”
• •“I do think trying to apply Kern's principles to a curriculum project is a good exercise.”
• •“I really liked being prompted throughout the year about your own project. It helped sharpen and refine it.”

After earlier sessions, learners did express a desire for more interactivity:

• •“Would be nice to have more interaction with colleagues, as opposed to independent study.”
• •“I think a reading assignment with a discussion at [the track] meetings might be more intellectually stimulating.”

In response to this feedback, the discussion assignments were divided into the two-part format. This allowed for more interaction and concrete feedback provided to learners. In future planning, we will consider having additional group discussions to augment faculty feedback on assignments, though time and scheduling constraints frequently prevent all learners from being able to be present for these discussions and risk eroding the spirit of the asynchronous nature of the exercise, which participants have noted to be favorable. Specifically in planning for the entering 2016–2018 class, we will add additional real-time group discussions following completion of the webinar reflection exercises based on this feedback. The goal of these additional sessions will be to continue augmenting the interactivity of the assignments while maximizing the ability of learners to participate. These discussions will likely take place in the form of an online communication platform such as Google Hangouts.

Discussion

Our goal was to expand the skill set of GME learners who were interested in developing skills as medical educators; specifically, we focused on curriculum development, as well as on an introduction to medical education theory and scholarship. As noted, there is a need to engage learners who are interested in becoming clinician-educators. When designing our project, we had two frameworks in mind: the limited time of the GME learner and the unique needs of the millennial learner. Because of the constantly varying clinical responsibilities of residents, requiring mandatory classroom attendance for multiple GME specialties would not be feasible. Furthermore, this generation of learners is nearly exclusively millennials; teaching recommendations for this generation center around appropriate use of technology to engage this group of learners.^5,6 To meet the nexus of these two demands, we created an asynchronous web-based curriculum so that learners could be engaged on a technological level, as well as budget their time by controlling the when aspect of their learning. Learners interested in this curriculum are a self-selecting cohort that is naturally interested in medical education. The web-based delivery allowed for participation of multiple GME specialties simultaneously.

Learners were given formative feedback 1 week after completion of every webinar-based assignment. As the curriculum evolved, we realized that the short-essay questions could often have multiple correct solutions, and we distributed accompanying answer keys in order to provide a framework to highlight all possible answers. Additionally, summary handouts for each webinar were developed to emphasize and provide clarity to core concepts. The goal of the evaluations was to reinforce basic concepts of medical education scholarship, learning theory, and curriculum design for future application to the learners’ own scholarly projects. Overall, the webinars met expectations and demands for a systematic approach to medical education curriculum, research, and scholarship. Furthermore, these webinars elicited a positive response from our learner group.

Applying the modified Kirkpatrick hierarchy as adapted by the Best Evidence Medical Education collaboration, this webinar series achieved excellent participation, which is interpreted as a Level 1 outcome, and garnered high satisfaction via evaluating attitudes of the learners (Level 2a).¹⁰ Modification of knowledge and skills (Level 2b) and behavioral change (Level 3) were also demonstrated by the application of faculty facilitator feedback to participating residents’ ELPs via completion of Part 2 of the reflection exercises. Areas for improvement and longitudinal follow-up of this curriculum will include addressing higher levels of the modified framework, such as change in organizational practice and benefits to society (Levels 4a and 4b), including eventual evaluation of participating residents’ ELP scholarship-related outcomes and further career participation in curriculum development and medical education overall. These longer-term outcomes will aid in evaluating efficacy of this curriculum.

We can identify one major limitation to the delivery of this curriculum: Providing meaningful feedback to each individual learner is time consuming for the faculty member. We aimed to provide feedback 1 week after completion of the webinar questions, which, depending on the group of learners, could be quite burdensome. However, we split the workload amongst multiple faculty facilitators (two faculty members per 11 learners) trained in medical education, and this limitation became very manageable. Additionally, depending on the preexisting curriculum, this course may not meet the needs of all individual institutions and GME programs. Despite these limitations, our pilot experience suggests that this resource will be valuable to clinician-educators who wish to train future leaders in medical education. Furthermore, the web-based asynchronous nature of this curriculum creates ease of use for busy GME learners and for simultaneous use by multiple specialties.

Appendices

A. Webinar 1.mp4

B. Webinar 2.mp4

C. Webinar 3.mp4

D. Webinar 4.mp4

E. Webinar 5.mp4

F. Webinar 6.mp4

G. Webinar 1 Summary Handout.pdf

H. Webinar 2 Summary Handout.pdf

I. Webinar 3 Summary Handout.pdf

J. Webinar 4 Summary Handout.pdf

K. Webinar 5 Summary Handout.pdf

L. Webinar 6 Summary Handout.pdf

M. Webinar 1 Discussion Packet.docx

N. Webinar 2 Discussion Packet.docx

0. Webinar 3 Discussion Packet.docx

P. Webinar 4 Discussion Packet.docx

Q. Webinar 5 Discussion Packet.docx

R. Webinar 6 Discussion Packet.docx

S. Course Director Packet.pdf

T. Sample Feedback Webinar 1.pdf

U. Sample Feedback Webinar 2.pdf

V. Sample Feedback Webinar 3.pdf

W. Sample Feedback Webinar 4.pdf

X. Sample Feedback Webinar 5.pdf

Y. Sample Feedback Webinar 6.pdf

All appendices are peer reviewed as integral parts of the Original Publication.