Engagement in Free Open Access Medical Education by US Nephrology Fellows

Visual Abstract

Abstract

Background

As free open access medical education (FOAMed) use increases, it is important to characterize how and why learners are using this educational material in nephrology. We describe the frequency, purpose, and type of FOAMed usage across US nephrology fellows.

Methods

In this cross-sectional survey, items were emailed to all US adult and pediatric nephrology fellows via the American Society of Nephrology (ASN) Fellow Survey in May 2022. The eight-item survey, developed to measure FOAMed engagement, had previously undergone instrument validation. The results were analyzed by descriptive statistics.

Results

In total, 43% (359/842) adult nephrology fellows and 51% (45/88) pediatric nephrology fellows completed the survey. Seventy-four percent (300/404) of fellows reported using FOAMed, and 72% (215/300) started using FOAMed within the past 2 years. Of FOAMed users, 41% (122/300) reported viewing FOAMed and 33% (99/300) reported applying knowledge gained from these resources daily or weekly. Common purposes for FOAMed engagement included searching Twitter to learn about others' opinions in the field (43%; 130/300), reading blogs to answer clinical questions (35%; 105/300), and listening to podcasts for the most up-to-date information (39%; 116/300). Compared with traditional educational resources, fellows preferred using FOAMed for staying up to date on nephrology topics (75%) and answering clinical questions (37%). Among all fellows, the greatest barriers to FOAMed use were unfamiliarity with FOAMed (27%; 111/404), validity concerns (22%; 90/404), and a lack of a local community of FOAMed users (22%; 87/404).

Conclusions

Seventy-four percent of nephrology fellows used FOAMed resources in a variety of ways, and of them, 33% of fellows clinically applied knowledge gained from these resources. Reasons for engaging with FOAMed varied across resources.

Introduction

In 2017, a national nephrology fellowship needs assessment survey identified only 55% of nephrology fellows as feeling fully prepared for practice after fellowship.¹ Although training programs have been charged with formal methods to enhance fellowship training, an online movement of freely available educational materials, known as free open access medical education (FOAMed), has been increasing in popularity within the nephrology community and beyond.^2–7 FOAMed has been defined as networks of blogs and microblogs, videos, podcasts, and other freely available medical resources⁵ in which the exchange of information and ease of accessibility around the world allows for collaboration.⁶ FOAMed resources include Twitter, Facebook, Instagram, blogs, YouTube, podcasts, and others. Examples of fellow-initiated FOAMed include Renal Fellow Network, ID Fellows Network, Liver Fellow Network, Cardionerds, and Hem-Onc Fellows Network. These continue to expand to more disciplines in internal medicine.

A survey of nephrology fellows conducted by the American Society of Nephrology (ASN) in 2021 demonstrated high use and similar perceived effectiveness of FOAMed compared with traditional educational tools, defined as nephrology journal articles and nephrology practice guidelines. The resource with greatest perceived effectiveness in the 2021 survey was UpToDate, an established digital resource of more than 25 years, now with a required subscription fee. When compared with a similar survey of nephrology fellows in 2016, the 2021 survey showed that FOAMed consumption increased.⁸ For example, NephJC use increased from 7% to 32%, Renal Fellow Network from 34% to 46%, and the American Journal of Kidney Diseases (AJKD) blog up from 7% to 15%. In addition, Twitter, NephMadness, Nephrology podcasts, GlomCon, and NephSIM, all not assessed in 2016, were used by 20% or more of fellows in 2021.⁸ Limited studies exist examining how these resources are being used by the fellows.⁸ Prior research has assessed single FOAMed resources focusing on implementation feasibility,^9,10 learner satisfaction,^11,12 and frequency of use,^13,14 yet there is a paucity of data demonstrating how learners are approaching the landscape of FOAMed in its entirety. Studies from the larger FOAMed community have called for additional research assessing FOAMed's effect on educational outcomes,^2,5,15–19 including connections to the international community,^20–22 accommodation to a variety of learning styles,^23,24 and shared teaching resources between programs.^25,26

To assess the impact of FOAMed on fellow education, we first need to better characterize the landscape of FOAMed usage among the fellows. By proactively understanding how fellows engage with FOAMed, we can address potential associated issues, such as misinformation, before they proliferate. We defined engagement, using the conceptual framework provided by Bond and colleagues,²⁷ including behavioral, affective, and cognitive engagement. According to Bond and colleagues, behavioral engagement is described as effort and persistence in activities by the learner, whereas affective engagement is described as emotional reactions to learning such as satisfaction and curiosity. Cognitive engagement is described as psychological investments made toward learning activities such as integration of ideas and teaching peers.²⁷ In this study, we addressed this critical gap in the literature by investigating in what ways and for what purposes current nephrology fellows are engaging with various FOAMed resources.

Methods

We performed a cross-sectional survey study using the ASN annual survey of all current nephrology fellows, both adult and pediatric. This optional survey seeks to help inform the society's efforts to foster the next generation of kidney health professionals and build the nephrology pipeline.²⁸

Survey Design and Instrument

Initial items to assess fellow FOAMed engagement were developed following literature review in fall 2021 by D.M.L. and M.A.S. using a construct modeling approach and Bond's engagement framework.^27,29 A total of 14 noncompulsory items consisting of multiple choice, ranking, and free response were pilot tested against a national group of junior attendings (private practice and academia) who had graduated fellowship within the past 3 years. Recruitment occurred in November 2021 via WhatsApp text link from preexisting groups (total responses=34). Using Rasch analysis, the quality of the measure along five strands of validity evidence, including content, response process, internal structure, external consistency, and fairness, was assessed.^29,30 To evaluate the validity evidence for internal structure, external consistency, and fairness for item functioning, we conducted item analysis using the item response theory. Item analysis using the Rasch modeling framework was performed using Berkeley Assessment System Software (BASS) to evaluate the item statistics.³¹ Reliability by internal consistency and alternate forms were considered high (weighted likelihood estimate=0.886 and 0.895, respectively). After completion of the pilot testing and validity assessment, we revised the FOAMed engagement instrument to an eight-item survey because of response process and internal consistency validity evidence. We submitted our FOAMed engagement instrument, comprising at least two items assessing each of Bond's facets of engagement.²⁷ It further underwent an iterative review process with the ASN Workforce and Training Data Subcommittee.

Design, Setting, and Participants

The final FOAMed engagement survey was distributed via BASS in May 2022 as a component of the ASN annual fellow survey. The FOAMed engagement survey items included in this analysis can be found in the Supplemental Material. Nephrology fellows were recruited through email and incentivized with prizes from ASN; no additional incentives were allocated from these investigators. The FOAMed engagement survey items were qualified as exempt (#1-35153) by the Institutional Review Board at University of California, San Francisco.

Analysis

Descriptive statistics were used to analyze quantitative results stratified by groups and reported as frequency distributions.

Results

Respondent Demographics

Overall, 404 of the eligible 930 nephrology fellows completed the survey (response rate=43%). This included 359 of 842 (42%) eligible adult nephrology fellows and 45 of 88 (51%) eligible pediatric nephrology fellows. Collected fellow demographic information is provided in Table 1. Although most (93%; 376/404) of the survey respondents were in their first or second year of fellowship training, they were broadly distributed across age ranges.

Table 1.

Nephrology fellows' demographic information

Characteristic	n (percent of survey respondents)
Fellowship type
Adult	359 (89)
Pediatric	45 (11)
Total	404
Fellow age, yr
<30	25 (6)
30–31	89 (22)
32–33	114 (28)
34–35	72 (18)
36–40	50 (12)
>41	44 (11)
NA	10 (3)
Fellowship year
First	189 (47)
Second	187 (46)
Third	24 (6)
Fourth or more	4 (1)
Fellowship program
Academic	378 (94)
Community	23 (6)
Unsure	3 (1)
Fellowship location
Northeast	134 (33)
South	117 (29)
Midwest	90 (22)
West	57 (14)
NA	6 (2)

Survey cohort included 842 adult nephrology fellows (response rate=43%) and 88 pediatric nephrology fellows (response rate=51%). NA, not available.

Fellows responded from across the United States, and 94% (378/404) were enrolled at academic programs.

Self-Identified FOAMed Use

Of the total 404 respondents, 300 fellows (74%) self-identified as having ever used FOAMed (Table 2). Congruent with the 2021 ASN survey results (Table 3), we found robust use of FOAMed content, yet unlike the year prior, respondents in 2022 reported a similar amount of use when stratified by age cohorts of ≤33 years and ≥34 years (77% versus 75%).⁸ The use of blogs in our survey (47%; 193/404) was similar to the 2021 survey where 47% reported using the Renal Fellow Network blog and 15% using the AJKD blog.⁸ Podcasts saw a considerable increase between years, from 24% reporting use of nephrology-focused podcasts and 9% general medicine–focused podcasts in 2021 to 49% (197/404) who reported listening to any podcasts in 2022.⁸ Similarly, Twitter use increased from 29% in the 2021 survey to 48% (194/404) in 2022.⁸ Just 29% of fellows identified as using NephMadness and 20% using NephSIM in 2021, whereas in 2022, 51% (207/404) stated they used any form of online games and cases, representing the greatest growth among the resource types we surveyed.⁸ The peak duration of FOAMed use among FOAMed users was 1–2 years (41%; 124/300). Fellows were also asked their perceptions of faculty use of FOAMed at their institution. Faculty were reported to never or rarely use FOAMed with lectures by 69% (279/404) of respondents, whereas only 7% (30/404) of fellow respondents described their faculty as using FOAMed with more than half or nearly all lectures.

Table 2.

FOAMed use among fellows and faculty lectures as reported by nephrology fellows May, 2022

Fellow-Reported FOAMed Use	N (%)
FOAMed use by fellows
FOAMed user	300 (74)
FOAMed nonuser	102 (25)
Did not respond	2 (<1)
Duration of fellow FOAMed use (among users), yr
<1	91 (30)
1–2	124 (41)
3–5	65 (22)
>5	19 (6)
Did not respond	3 (1)
FOAMed use by faculty in lectures
Never	115 (29)
Rarely	164 (41)
< half of lectures	86 (21)
> half of lectures	25 (6)
Nearly all lectures	5 (1)
Did not respond	9 (2)

Nearly three quarters of fellows reported using FOAMed, while 70% of faculty were reported as never or rarely using FOAMed in their lectures. FOAMed, free open access medical education.

Table 3.

A comparison of self-reported FOAMed resource use between 2021 and 2022 ASN surveys

FOAMed Resource	2021 Use, n=501	2022 Use, n=404
Blogs		48% (193)
Renal fellow network blog	47% (235)
AJKD blog	15% (77)
Podcasts		49% (197)
Nephrology-focused podcasts	24% (118)
General medicine–focused podcasts	9% (43)
Twitter	29% (145)	48% (194)
NephJC (Twitter-based journal club)	33% (166)
Interactive games/cases		51% (207)
NephMadness	29% (144)
NephSIM	20% (98)

Notably comparison is limited by the style in which fellows were queried about use and the high turnover of nephrology fellows responding to the survey each year.⁸ FOAMed, free open access medical education; AJKD, American Journal of Kidney Diseases.

Describing Engagement with FOAMed

Fellows who self-identified as FOAMed users (n=300) were asked how and how often they engaged with FOAMed material in the past 2 months (Figure 1). These included viewing (or listening to) FOAMed, applying knowledge gained from a FOAMed resource to clinical practice, interacting with (i.e., reposting, commenting on, or sharing) a FOAMed post, and composing original content to share/teach other FOAMed users. As shown in Figure 1, a majority of fellows viewed FOAMed resources (89%; 268/300) and applied knowledge gained into clinical practice (86%; 258/300) at least monthly, and one third of fellows interacted with FOAMed posts (39%; 118/300) or shared original content via these resources (32%; 96/300). On a daily or weekly basis, 41% of fellows (122/300) viewed FOAMed resources, and 33% of fellows (99/300) applied knowledge gained from these resources to answer clinical questions.

Figure 1.

Self-reported frequency of various types of free open access medical education (FOAMed) engagement by fellows who reported FOAMed use (n=300). As difficulty of engagement increases (from simply viewing a FOAMed resource to creating and posting original FOAMed content), the frequency with which fellows are engaging in that way decreases as expected.

We next asked fellows to identify all the reasons they used a particular FOAMed resource (Figure 2). FOAMed resources were divided into four main categories: blogs, podcasts, Twitter, and interactive games or cases. Twitter was accessed most frequently for the widest range of desired intent, as is shown in Figure 2. Interactive games and cases were least used by fellows for each prespecified indication for FOAMed use. Blogs, closely followed by Twitter, were used most for answering clinical questions (35%; 105/300), whereas podcasts were heavily used for their ability to multitask while listening (37%; 112/300).

Figure 2.

Self-reported reason for FOAMed use by type of FOAMed resource among fellows reporting FOAMed use (n=300). Notably, games and cases were used less frequently overall. Fellows used Twitter with high frequency for many different reasons, whereas podcasts were specifically used for fellows' ability to multitask, and blogs were popular for seeking answers to clinical questions.

Fellows responded with a range of opinions when asked on what occasions they preferred FOAMed to traditional resources (Figure 3). Most respondents favored FOAMed resources for staying up to date (75%; 224/300). Few FOAMed users (9%; 27/300) did not have any preference for FOAMed over traditional learning resources, defined as the absence of selecting a preference for FOAMed. Stated alternatively, 90% (273/300) of fellows using FOAMed prefer FOAMed resources to what they are learning by traditional formats for at least one purpose.

Figure 3.

Reasons why fellows preferred using FOAMed resources instead of traditional learning resources, among fellows who use FOAMed (n=300). Fellows were allowed to select more than one reason. Those with no preference for FOAMed did not select any of the reasons above.

Barriers to FOAMed Engagement

Seventy percent of all nephrology fellows (281/404) reported significant barriers to FOAMed use (Figure 4). We stratified barriers by self-reported FOAMed use and found that 64% (67/104) of FOAMed nonusers and 71% (214/300) of FOAMed users identified at least one concern for using FOAMed. When asked to describe these barriers, FOAMed users reported greatest concerns with validity (28%; 84/300), unfamiliarity with resource (26%; 77/300), lack of a local community of FOAMed users (23%; 70/300), and technology (19%; 56/300). Within the group of FOAMed nonusers, fellows identified an unfamiliarity with FOAMed resources as the most common barrier (33%; 34/104), and the lack of a local community of FOAMed users as the second most common barrier (17%; 17/104), while only 6% (6/104) and 7% (7/104) reported validity and technology concerns, respectively. No concerns, defined as the absence of responding to the item, were reported in 29% (86/300) of FOAMed users and 34% (35/104) of nonusers.

Figure 4.

Fellow-reported barriers to FOAMed use stratified by self-identified FOAMed users (white) and FOAMed nonusers (gray). Fellows could select multiple barriers, and percentages displayed are reported within their respective group (user versus nonuser), hence the sum may be >100%. Nonusers had a greater relative percentage of unfamiliarity with resources and concerns regarding the lack of data for FOAMed as an educational tool. Overall, contextual factors, including difficulty with technology, a lack of familiarity with FOAMed resources, and no local community of FOAMed users, comprised the greatest proportion of perceived barriers.

Discussion

In this survey-based quantitative study of self-identified FOAMed engagement among current nephrology fellows, we built on prior research by describing how and why fellows are using FOAMed.^6,8,20,32 We found that 74% (300/404) of fellows reported using a FOAMed resource in their training with roughly equal distribution across various FOAMed types (Table 3). The increased FOAMed use seen across fellows when compared with 2021 survey data must be interpreted cautiously. Notable differences in data collection between surveys may account for some of these differences, including the grouping of FOAMed resources by general type instead of unique entities (such as NephJC or NephMadness), the inclusion of non-nephrology FOAMed use, and self-identification of FOAMed resources the fellow did not use (as opposed to ones used). Furthermore, from our demographic data, 53% (215/404) of fellows started using FOAMed in the past 0–2 years and 47% (189/404) of fellows were in their first year of fellowship. Taken together, these factors may limit the degree to which we can compare the 2022 data directly with 2021; however, our current study suggests persistent use of FOAMed resources by nephrology fellows.

Unique to this study, fellow FOAMed users were asked to describe patterns of FOAMed use by frequency and characterization of interaction (Figure 1). We were unable to find comparison data on frequency of accessing non-FOAMed educational resources by nephrology fellows or medical trainees at large for comparison; however, data from the 2021 ASN survey found that 82% of fellows reported using UpToDate overall.⁸ When it came to interacting with FOAMed, defined as reposting, liking, or commenting on another's post, we found a majority of fellows were passive observers of FOAMed content. This is congruent with the use of social media by health care professionals as described by Rolls and colleagues who found that most virtual community members had low posting behaviors but more frequently read or accessed educational content.³³ We confirmed the presence of what has been termed lurkers, FOAMed users who read content but do not contribute to discussions or the body of knowledge. The pattern of FOAMed use in our study suggests that even without interacting or contributing to FOAMed, learners are able to take away valuable pearls for application in the clinical domain. Although more difficult to track, lurkers comprise a prominent population of learners that must be included in future analyses of FOAMed's educational effect.³⁴ Several studies have assessed the positive impact of interaction within the online community from a learners' perspective,³⁵ yet there remains relatively little assessment on the larger community of FOAMed viewers despite their known prevalence.³⁶

Fellows also varied their FOAMed use by the type of resource (Figure 2). Podcasts were most used for their ability to multitask and remain current in the field. This is not unique to nephrology fellows. In a 2021 prospective survey on an emergency medicine–based podcast, Roland found learners used podcasts specifically for the ability to multitask, in addition to learning core material and refreshing knowledge.³⁷ Blogs, in contrast, were most used by fellows to answer clinical questions and learn about others' opinions. With the increased availability, searchability, and the number of contributors to these knowledge repositories, it is perhaps not surprising that FOAMed users reached for blogs for these purposes.^4,6,13

Twitter was the most commonly used FOAMed resource across all options, except ability to multitask. One could argue that Twitter is simply the most prevalent form of FOAMed, especially within the nephrology community and thus has more opportunities to be used for each purpose. In our study, usage rates were similar across groups, and even in the 2021 survey, Renal Fellow Network had 58% fellow use while Twitter had only 28% use.⁸ As an alternative explanation, published studies and commentaries across the past decade highlight the versatility of Twitter as an educational tool, citing it as a facilitator of communities of practice for knowledge translation,³⁸ a stimulator of learners' preconceptions,³⁹ and a builder of professional identity.⁴⁰ Among nephrology fellows in our survey, Twitter was most frequently used to learn about others' opinions, remain current in the field, and build connections, the last of which was unique to Twitter.

Fellows expressed fewer preferences for using interactive games and online cases, despite an increase in use compared with the 2021 survey.⁸ One plausible explanation is that interactive games and simulations are often used in a prescribed manner (i.e., one is instructed to submit a NephMadness bracket every March or often sets aside a discrete amount of time to complete a NephSIM case), as opposed to the sporadic and independent use initiated by the fellow for other types of FOAMed resources. In addition, the timing of the survey, more than 2 months removed from NephMadness and at the end of an academic year when motivation to complete simulated cases may be lower, may also have affected the responses in this category. Regardless, fellows are clearly using a variety of types of FOAMed in unique ways and with intentional purposes.

Compared with more traditional learning resources, fellows preferred FOAMed resources most to stay up to date on nephrology topics (Figure 3). Several prior studies also identified timeliness as a significant benefit of the FOAMed space.^14,38,41 Although more fellows preferred using FOAMed for answering clinical questions rather than for sharing knowledge or experiences, it is difficult to discern from this survey alone if this is a causative factor for the self-reported FOAMed use patterns (Figure 1) or merely resultant from the sheer greater numbers who are viewing versus contributing to FOAMed. Additional qualitative studies aimed at investigating reasons behind FOAMed uptake, engagement, and preference for use will be critical to guide both educators and learners toward successful integration of these resources into fellows' learning.

Because of the popularity and direct patient care implications with FOAMed use, it is imperative that FOAMed creators ensure content is rigorous, referenced, and balanced.

It is unknown what factors are barriers to using FOAMed. We found a majority of all fellows reported barriers with FOAMed use. Paradoxically, FOAMed users were more likely than nonusers to cite a difficulty with technology as a barrier to further FOAMed engagement. This may be related to the uncovering of additional technical difficulties with ongoing use, as well as an internal comparison in perceived adeptness with technology against other FOAMed-using peers. A greater perceived barrier for FOAMed users compared with nonusers was also found for concerns regarding the validity of information and privacy. We hypothesize that this is related to the uncovering of additional concerns with increased use. It further suggests that despite these concerns, users continue to access FOAMed resources. Future research could examine whether this is because of work-arounds to perceived limitations, a building of tolerance to dislikes, or other reasons. This also further reiterates the need for FOAMed creators to ensure content is current, valid, well referenced, and without bias.

Our study has limitations. It was conducted via a single survey of fellows' perceptions and had a response rate of 43% (404/930), potentially capturing a greater percentage of FOAMed users who would be more apt to complete the survey. We were further limited by the use of multiple choices and select all that apply items, which restricted fellows' responses. In addition, we did not assess comparison of barriers to FOAMed with barriers to more traditional resources, including fee-based, digital resources, and thus are unable to fairly characterize the extent of the effect of these barriers to FOAMed specifically. Strengths of our survey include its dissemination to all fellows via the ASN annual survey as opposed to hosting the survey on a FOAMed platform, which would bias the results. Pilot testing of the items among a group of recently graduated nephrology attendings with rigorous validity testing at the item and instrument level was another strength of our study.

The future of FOAMed is unfolding before us, and we must seek to understand and incorporate the views of the end user: the learner.^3,9,42–45 To leverage FOAMed as an educational tool, we need to address concerns of misinformation by proactively building on our current understanding of the FOAMed landscape and educating fellows to be wise consumers of FOAMed.

Supplementary Material

cjasn-18-573-s001.pdf ^{(86.6KB, pdf)}

Disclosures

D.M. Larsen reports employment with San Francisco Veterans Affairs Medical Center and is a writer for NephMadness, an online “nephrology educational initiative that is modeled after the college basketball tournament but with nephrology concepts.” This is a voluntary role for 2022–2023. M.A. Sparks reports employment with Durham VA Health Care System, research funding from Renal Research Institute, and honoraria from Elsevier—Nephrology Secrets. M.A. Sparks is co-founder and on the executive committee of NephMadness and receives a stipend from AJKD for this role. M.A. Sparks is on a board of directors of NephJC (a volunteer role), a nonprofit organization committed to enhancing FOAMed in nephrology. M.A. Sparks is a faculty lead of the Renal Fellow Network, is a member of The Nephron Segment Podcast, and is an advisory board member of NephSIM—all voluntary roles. M.A. Sparks is a member of the American Board of Internal Medicine Nephrology Board and receives a stipend for this role. M.A. Sparks is on the editorial boards of American Journal of Kidney Diseases, ASN Kidney News, CJASN, Kidney Medicine, and Kidney360. M.A. Sparks reports advisory or leadership roles for KCVD Membership & Communications Committee—AHA; KCVD Scientific & Clinical Education Lifelong Learning Committee (SCILL)—AHA; and NKF North Carolina—Medical Advisory Board. The remaining author has nothing to disclose.

Funding

None.

Author Contributions

Conceptualization: Dana M. Larsen.

Data curation: Dana M. Larsen.

Formal analysis: Dana M. Larsen, Matthew A. Sparks.

Investigation: Dana M. Larsen, Matthew A. Sparks.

Methodology: Christy K. Boscardin, Dana M. Larsen, Matthew A. Sparks.

Supervision: Christy K. Boscardin, Matthew A. Sparks.

Writing – original draft: Dana M. Larsen.

Writing – review & editing: Christy K. Boscardin, Matthew A. Sparks.

Supplemental Material

This article contains the following supplemental material online at http://links.lww.com/CJN/B646.

Supplemental Material. Final FOAMed Engagement Survey Items.