Why Not a Podcast? Assessing Narrative Audio and Written Curricula in Obstetrical Neurology

Abstract

Background

Medical podcasts have the potential to educate residents and fellows in specialized or uncommon disciplines, but the acceptability and benefits of educational podcasts are unclear.

Objective

We compared knowledge acquisition and engagement of audio-only (podcast) versus written curricular formats and assessed podcast feasibility and uptake for teaching obstetrical neurology to residents and fellows.

Methods

Key concepts in obstetrical neurology were developed into parallel case-based modules: written reviews and podcasts interwove patient and expert voices with narration. In 2017, we tested this curriculum among 60 volunteer residents and fellows in obstetrics and gynecology, neurology, emergency medicine, internal medicine, and family medicine training programs at a single institution. Participants took content-based pretests, were randomized, and then completed written (n = 32) or podcast (n = 28) modules, and finally, completed posttests and feedback questionnaires.

Results

Among all participants, there was an increase in immediate posttest scores compared with pretest scores (46 of 60, 77% ± 17% pretest versus 56 of 60, 93% ± 10% posttest, P < .05), with participants in the podcast and written groups performing equally well. However, listeners rated the podcasts somewhat higher than written materials in the areas of maintaining interest, enjoyability, entertaining, and desire for wider use.

Conclusions

Written and podcast curricula improved immediate knowledge similarly, but the narrative-style podcasts were perceived as more enjoyable by residents and fellows from several specialties, suggesting narrative podcasting can be an engaging and feasible educational alternative for trainees to acquire information.

What was known and gap

Medical podcasts are increasingly popular, but the acceptability and benefits of using them to educate medical learners are unclear.

What is new

A comparison of knowledge acquisition and engagement between a podcast and written curricula.

Limitations

Small sample size of volunteers at one institution and limited follow-up.

Bottom line

Written and podcast curricula provided similar immediate knowledge acquisition, but the podcasts were more enjoyable for residents and fellows from several specialties.

Introduction

Without a doubt, mobile technology has influenced the way medical residents and fellows learn.^1–3 Medical education podcasts have increased in popularity among learners in spite of minimal evidence of efficacy and value to date.⁴

Podcasts can enhance asynchronous learning in a blended classroom model or communicate adjunctive curricular information, especially in topics that fall outside typical residency or fellowship didactics, including the diagnosis and treatment of neurological diseases in pregnancy (obstetrical neurology), an uncomfortable knowledge gap for neurologists, obstetricians, and emergency and primary care practitioners, in which no standardized residency or fellowship curriculum exists.

We hypothesized that a narrative podcast-based curriculum in obstetrical neurology would provide similar increases in content-specific knowledge to residents and fellows compared with written cases but would be rated a superior learning platform on feedback questionnaires.

Methods

Participants and Setting

The recorded podcast interviews and impact study took place at hospitals affiliated with the Warren Alpert Medical School of Brown University (AMS) in 2016–2017. All participants were volunteers from AMS residencies and fellowships (neurology, obstetrics and gynecology, family medicine, internal medicine, and emergency medicine). The AMS has over 100 residency and fellowship programs with 696 residents, but we selectively recruited through contact with program directors in fields pertinent to obstetrical neurology.

Curricular Development

In 2016–2017, one author (J.R.) acquired the equipment and proficiency to make podcasts (0.2 full-time equivalent [FTE]). She developed a standardized approach to medical podcasting, modeled after the “NPR-style” audio format familiar to listeners of news and entertainment podcasts, which she teaches in seminars and workshops to AMS faculty and students. Briefly, this approach incorporates elements, including an introductory “hook,” a “roadmap” (outline) with embedded learning objectives, “signposts” (segment summaries), a narrative arc, a casual tone, differing voices and perspectives through interviews, music, and good sound quality, and editing. Two authors (J.R. and M.H.) collaborated to produce the podcasts for this project with a Marantz (Kawasaki, Japan) recorder, a Shure (Miles, IL) microphone, and Hindenburg (Copenhagen, Denmark) editing software. Audio and written curricula were posted on a Squarespace (New York, NY) website (https://www.neurostories.com), and podcasts are available as The Push: A Pregnancy Neurology Podcast on Apple (Cupertino, CA) Podcasts.⁵

Two authors (J.R. and N.M.) developed a clinical case series in obstetrical neurology, matching real patient scenarios with literature reviews, as standard written case studies (1000 to 3000 words), in parallel with separate, high-quality, audio-only podcasts that adopted a narrative storytelling approach, interweaving patient and expert interviews (15 to 20 minutes) with voiceover narration. Case details, learning objectives, and references were identical between written and audio modules (written cases were not simply transcripts of the podcasts). Knowledge experts vetted the modules. All recorded patients provided detailed written consent, used pseudonyms with no potentially identifying information, and were given full access to the podcasts prior to release.

Study Design

Two authors (J.R. and N.M.) contacted program directors and arranged 6 optional, 1-hour resident and fellow conferences to demonstrate each curriculum. Each conference covered only one illustrative obstetrical neurology topic: Stroke in Pregnancy (4 conferences and cohorts) or Migraine in Pregnancy (2 conferences and cohorts), to demonstrate podcast feasibility and replicability. Residents and fellows who volunteered for the study were given $5 gift cards. Participants completed content-specific pretests, were randomized to either the podcast or written curriculum, and then separated by group into 2 rooms to complete the curricular module (15 to 20 minutes). Upon module completion, all participants completed posttests and feedback questionnaires (detailed below).

Outcomes

We measured knowledge with a pre- and posttest study design and acceptability with a questionnaire. Pretests and posttests consisted of separate samplings of 6 to 7 module content-specific (stroke or migraine) multiple-choice questions based around learning objectives and a 16-question feedback questionnaire (provided as online supplemental material). The questionnaire asked participants to rate their prior knowledge of the topic they learned and their prior knowledge and interest in obstetrical neurology on a 5-point Likert scale, (1, not very or not at all, to 5, very or most). Participants rated the curricula (1, strongly disagree, to 5, strongly agree) on subjective descriptors (increasing interest or knowledge and enjoyability among others) and provided comments. The knowledge tests and survey were not pretested or otherwise evaluated for validity evidence.

This study was exempted by the AMS Institutional Review Board.

Analysis

To compare knowledge (posttest) improvement before and after curricular exposure, repeated measures analysis of variation was performed. Factors such as prior knowledge, interest, and specialty were included to determine main and interaction effects with the curriculum type (6 comparisons). To compare experiential outcomes and favorability between curricular types, we performed ordinal regression and adjusted for the following factors: prior specific knowledge of the subject area, prior knowledge of obstetrical neurology in general, prior interest in the topic, medical specialty, and case type. Analysis was performed with SPSS software (version 24.0, IBM Corporation, Armonk, NY), and P < .05 was considered statistically significant.

Results

Of the 60 residents and fellows who attended optional in-person conferences across 6 dates, all 60 volunteered to participate in the study. Specialty fields included neurology (n = 19), obstetrics and gynecology (n = 11), internal medicine (n = 13), emergency medicine (n = 1), and family medicine (n = 16). Conferences were topic-specific, and included 4 conferences and cohorts reviewing stroke (39 participants) and 2 conferences and cohorts reviewing migraine (21 participants). A total of 28 participants (20 stroke and 8 migraine) were randomized to the podcast groups and 32 (19 stroke and 13 migraine) to the written groups. Scores on the pre- and posttests were compared (percentage correct) with a Student's t test. With the Student's t test, the percentage of correct scores on the pre- and posttests showed an increase in posttest scores in all groups (46 of 60, 77% correct ± 17% pretest versus 56 of 60, 93% correct ± 10% posttest, P < .05), with podcast and written groups performing equally well based on pre- or posttest scores, regardless of self-reported prior knowledge level of the specific topic or general field or prior interest in the topic or medical specialty (Table 1).

Table 1.

Descriptive Statistics for Knowledge Improvement

Descriptor and Likert Scorea	Test Scores (of 1.00 ± 95% Confidence Interval)		P Value
	Precurriculum	Postcurriculum
Student's t test	0.77 ± 0.17	0.93 ± 0.10	.048
Prior self-assessed knowledge of topic			.78
1	0.72 ± 0.13	0.91 ± 0.13
2	0.67 ± 0.15	0.89 ± 0.13
3	0.82 ± 0.16	0.94 ± 0.07
4	0.79 ± 0.17	0.97 ± 0.07
5	1.00	0.86
Prior self-assessed knowledge of obstetrical neurology			.71
1	0.68 ± 0.09	0.97 ± 0.06
2	0.78 ± 0.19	0.92 ± 0.11
3	0.75 ± 0.17	0.93 ± 0.10
4	0.79 ± 0.14	0.93 ± 0.08
5	0.92 ± 0.12	1.00 ± 0.00
Prior interest in obstetrical neurology			.76
1	0.78 ± 0.10	1.00 ± 0.00
2	0.70 ± 0.18	0.93 ± 0.12
3	0.78 ± 0.16	0.90 ± 0.10
4	0.76 ± 0.18	0.94 ± 0.10
5	0.80 ± 0.15	0.92 ± 0.08
Specialty			.32
Neurology	0.85 ± 0.15	0.96 ± 0.06
Internal medicine	0.69 ± 0.16	0.89 ± 0.10
Obstetrics and gynecology	0.77 ± 0.17	0.91 ± 0.12
Family medicine	0.75 ± 0.16	0.93 ± 0.10
Emergency medicine	0.57	1.00
Curriculum used			.65
Podcast	0.76 ± 0.17	0.93 ± 0.09
Written	0.77 ± 0.17	0.93 ± 0.10
Case			.86
Stroke in pregnancy	0.76 ± 0.18	0.92 ± 0.10
Migraine in pregnancy	0.78 ± 0.14	0.95 ± 0.09

^aParticipants were asked to rate their prior knowledge of the topic and field and their prior interest on 5-point Likert scales (1, not very or not at all, to 5, very or most).

On the feedback questionnaire, the podcasts were rated somewhat higher than written cases in the areas of maintaining interest, enjoyability, entertainment, and wanting to listen to them more often in medical education (provided as online supplemental material). Other ratings (sparking interest, providing satisfaction, or using it again in one's field) were similar between the podcast and written groups. There was no difference in ratings for self-assessed improvement in skills and knowledge. Written comments about the podcasts were generally very positive (Tables 2 and 3).

Table 2.

Feedback Questionnaire: Descriptive Statistics for Experience Ratings by Curriculum Type

Descriptor and Likert Scorea	Module Average Score (95% Confidence Interval)		P Valueb
	Podcast (n = 28)	Written (n = 32)
Maintained interest	5 (4–5)	4 (4–4.75)	.001
5	19	8
4	7	17
3	1	6
2	0	1
1	0	0
NR	1	0
Sparked interest	5 (4–5)	5 (4–5)	.15
5	14	11
4	10	14
3	3	5
2	0	1
1	0	1
NR	1	0
Improved skills/knowledge	5 (4–5)	4.5 (4–5)	.32
1	17	16
2	10	15
3	0	1
4	0	0
5	0	0
NR	1	0
Enjoyable	4 (4–5)	4 (3–5)	.001
5	18	9
4	9	11
3	0	11
2	0	0
1	0	1
NR	1	0
Entertaining	5(4–5)	3(2.25–4)	< .001
5	17	3
4	9	10
3	1	11
2	0	4
1	0	4
NR	1	0
Provided satisfaction	5(4–5)	4(3.25–4.75)	.06
5	14	8
4	11	16
3	2	5
2	0	2
1	0	1
NR	1	0
Would use it again in my field	5 (4–5)	4 (4–5)	.12
5	17	13
4	8	12
3	2	3
2	0	4
1	0	0
NR	1	0
Want to see used more often in medical education	5 (5–5)	4.75 (3.5–5)	< .001
5	19	8
4	6	8
3	2	7
2	0	5
1	0	4
NR	1	0
Motivated to create similar educational modules	5 (4–5)	4 (3.5–5)	.19
5	12	7
4	5	9
3	3	8
2	6	5
1	1	3
NR	1	0
Increased confidence about obstetrical neurology, N (%)	28 (100)	29 (91)	.99

Abbreviation: NR, not rated.

^aParticipants were asked to rate a series of statements about the educational program they experienced on a 5-point Likert scale (1, strongly disagree, to 5 strongly agree); the questionnaire is online as supplemental material.

^bValues adjusted for prior specific knowledge, prior general knowledge, prior interest, specialty, prior survey exposure, and case.

Table 3.

Representative Feedback Comments

Podcast Group	Written Group
“Thoughtful, engaging, and extremely high-yield! Awesome job!”	“The reading was well written, but I wish I could have had the podcast—I love them for learning!”
“Podcast was extremely well done. Educational, entertaining, and seemed to be appropriate across multiple specialties/levels of training.”	“I am used to the written format and will likely continue to use it for much of my education, but the podcast format (when used in the past) was very engaging and piqued my interest in a topic that was not previously particularly interesting to me.”
“Very enjoyable! Like Serial but with more learning!”	“In general, I prefer oral/podcast/conferences to written information, but if a naive or rare topic came up, on which I had not taken notes at a previous conference, I would usually use written reviews or online resources.”

For each podcast, time spent interviewing, scripting, narrating, editing, and producing each episode varied. The first podcast took approximately 20 hours for a novice to produce; subsequent podcasts took 4 to 10 hours. Of the study podcasts, Stroke in Pregnancy (15 minutes) and Migraine in Pregnancy (20 minutes) required approximately 20 hours and 10 hours to produce, respectively. Curricular content development time (eg, literature search, concept, and writing) and podcasting skills development were incorporated into the lead author's 0.2 FTE. Material costs for the project (recording equipment and software) were less than $1,000. Since its initial posting in 2017, this podcast curriculum has garnered over 16 000 listeners worldwide on 6 continents, based on Squarespace and Apple analytics.

Discussion

In this study, a high-quality narrative podcast curriculum in obstetrical neurology provided equivalent immediate increases in content-specific knowledge for multispecialty residents and fellows when compared with a written curriculum and was perceived as a more engaging learning experience.

This is one of the first studies to compare well-produced narrative podcast-based educational modules to standard written case approaches for the same learning objectives.

Clinician-educators lacking podcast knowledge might anticipate higher opportunity cost and time. Podcasts are relatively inexpensive to produce and disseminate (less than $1,000). Inexpensive (or free) podcasting can be done with a smartphone, a computer, free or institutionally provided editing software, institutional audiovisual resources, and web hosting. Although creating engaging modules takes time, we created supplementary resources specific to medical podcasting: a free open-access medical podcasting manual is published online.⁵ After the study, curricula were posted online, free and open to the public at https://www.neurostories.com. Excellent medical podcasts are produced by busy medical students, residents, and recent graduates (eg, The Short Coat [theshortcoat.com], CREOGs over Coffee [https://creogsovercoffee.podbean.com], and The Curbsiders [https://thecurbsiders.com]), further supporting the feasibility for clinician-podcasters. One critique of podcasts may be the lack of quality-assurance measures, leading to a recent effort to identify and promote specific quality indicators among digital education formats.⁶ Our curricular content was based on peer-reviewed articles; all podcasts were reviewed by field experts prior to distribution.

Prior studies demonstrated that podcast formats were felt by students or medical trainees to be helpful for review purposes, but in some cases less engaging or enjoyable than standard teaching formats.^7,8 However, these studies compared live lectures to their own digital recordings, instead of a novel, independent educational digital format. Our podcasts used a narrative style that melded patient and expert voices to enhance case-based education; we hypothesized that high production values would increase learner engagement, especially for a nonrequired topic, and high rating scores supported that finding.

Limitations of our study included a small sample size of volunteers, a single institution, and an artificial learning environment in which podcasts were played aloud for groups of learners in a classroom rather than individually on mobile devices in private. These factors will limit generalizing the findings to nonvolunteers and more typical podcast listening environments. A more important limitation is the lack of long-term follow-up. Immediate knowledge improvement occurred in the study, but it is not known how written versus podcast materials affect long-term knowledge retention or future resident behaviors with patients.

After public release of the curricula, broad dissemination and uptake was noted through online analytics. A large online cross-institutional study that includes learner demographics might corroborate this study's findings as would providing further information about long-term information retention, identifying learners most likely to benefit, and studying the effect on low-resource areas in which access to specialists or specialty-based education may be otherwise limited.

Conclusions

Among residents and fellows, our narrative case-based podcasts were deemed a more engaging and enjoyable learning experience that provided similar knowledge compared with standard written cases, even when controlling for factors including prior interest and expertise in the topic.