Systematic online academic resource (SOAR) review: Endocrine, metabolic, and nutritional disorders

Abstract

Background

Free open‐access medical education (FOAM) has become an integral resource for medical school and residency education. However, questions of quality and inconsistent coverage of core topics remain. In this second entry of the SAEM Systematic Online Academic Resource (SOAR) series, we describe the application of a systematic methodology to identify, curate, and describe FOAM topics specific to endocrine, metabolic, and nutritional disorders as defined by the 2016 Model of the Clinical Practice of Emergency Medicine (MCPEM).

Methods

We developed an automated algorithm to search 264 keywords derived from nine subtopics within the MCPEM category in the FOAM Search (a customized FOAM search tool) and the Social Media index. The top 100 results were extracted for each keyword. Resources underwent a manual iterative screening process, and those relevant to endocrine, metabolic, or nutritional disorders and EM were evaluated with the revised Medical Education Translational Resources: Impact and Quality (rMETRIQ) tool.

Results

The search yielded 36,346 resources, of which 756 met the criteria for quality assessment. After rMETRIQ tool training, four raters demonstrated an average measured intraclass correlation coefficient of 0.94 (95% confidence interval = 0.88 to 0.97, p < 0.001). A total of 121 posts (16% of posts) covering 25 subtopics were identified as high quality (rMETRIQ ≥16). The most covered subtopic was potassium disorders, representing 15% of all posts. Subtopics that did not have a high‐quality resource identified include metabolic alkalosis, respiratory alkalosis, fluid overload, phosphorus metabolism, hyperglycemia, malabsorption, malnutrition, and thyroiditis. From most to least common, the overall target audience was junior resident (91%), PGY‐1 resident (88%), senior resident (81%), clerk (64%), attending (50%), and preclerkship (9%).

Conclusions

We systematically identified, described, and curated FOAM resources for EM learners on the topic of endocrine, metabolic, and nutritional disorders. A final list of high‐quality resources can guide trainees, educator recommendations, and FOAM authors.

INTRODUCTION

Free open‐access medical education (FOAM) resources in emergency medicine (EM) are routinely used informally via independent resident curation and formally as part of local and national residency curricula. ¹ , ² , ³ , ⁴ , ⁵ , ⁶ , ⁷  The abundance of these online resources and the diversity in platforms and formats make it difficult to compare and assess the quality of the information presented. ⁸ , ⁹ , ¹⁰ , ¹¹ Furthermore, FOAM has limited comprehensiveness in addition to being entirely decentralized, which can makes navigation complicated. ¹² , ¹³  These factors plus the enormous volume of FOAM produced makes curation extraordinarily difficult on the local level. Thus, the need exists for a systematic search of FOAM content to produce a curated list of resources organized by subtopic and learner level.

Here, we present the second in a series that aims to create a new type of synthesis scholarship that borrows from the systematic review format to aggregate and filter high‐quality, online academic resources for the purposes of creating a centralized resource ¹⁴ (please see Box 1 for details of this initiative).

BOX 1. The Objective of the SAEM Systematic Online Academic Resource (SOAR) Topic Review Series.

This is the second entry in the SAEM Systematic Online Academic Resource (SOAR) Topic Review series. We aim to systematically identify online resources by topic to determine FOAM’s current coverage of EM topics and subtopics, assess the quality of these resources with a validated tool, and collate links to identified high‐quality online resources in a user‐friendly framework. Thus, the SAEM SOAR Topic Reviews will enable all users (individual learners, educators, and program leaders) to easily find relevant, high‐quality online resources. Additionally, this ongoing series will help create a durable scientific evaluation of FOAM in EM.

This second entry in the SAEM Systematic Online Academic Resource (SOAR) Topic Review series identifies and evaluates online education resources related to endocrine, metabolic, and nutritional disorders. Future editions will continue to review other topics. This article and its methodology can serve as a guide to future authors for future topics (and, eventually, updated reviews of prior topics) to provide a comprehensive description of the FOAM landscape.

METHODS

Study design

As in the first SOAR publication, the design of this review was conducted similar to that of a traditional systematic literature review. We again attempted to adhere to the PRISMA guidelines as closely as possible. ¹⁵

Topic identification

Appropriate subtopic searches were based on the 2016 American Board of Emergency Medicine's Model of the Clinical Practice of Emergency Medicine (MCPEM) document because it “represents essential information and skills necessary for the clinical practice of EM by board‐certified emergency physicians.” ¹⁶ , ¹⁷ For the “endocrine, metabolic, and nutritional” disorders search, we used the MCPEM topic's nine headings and over 30 subheadings to create a list of keywords, including synonyms, for the search. In addition, the authors also reviewed signs, symptoms, and presentations of the MCPEM topic and subtopics. The final search included 264 keywords (see Table S1).

Database search

We selected FOAM Search (a customized FOAM search tool; GoogleFoam.com) and the top 50 FOAM websites per the Social Media index (SMi) as our two search repositories to identify online educational resources targeted to health professionals. ¹⁸ , ¹⁹ , ²⁰  This strategy was chosen to avoid capturing irrelevant content intended for lay public consumption that would have resulted from a generic engine (e.g., Google). We restricted our search to the top 50 sites on the SMI (SMI‐50) based on input from the prior SOAR review authors, who observed that sites ranked lower than 50 were generally of less high quality in addition to resource limitations.

With the use of a novel software program, each term was entered into FOAM Search and the search engines of each site from the SMI‐50 list, and the program automatically extracted and collated the top 100 results for every keyword searched. The results of the initial search were extracted with their page title, keyword, source type (journal, blog, podcast, archive), and URL. Of note, one author (JH) performed a manual review of sites listed and discovered a flaw in the automated search of one site and found 22 posts initially missed.

Inclusion criteria

All open‐access educational resources listed on FOAM Search and found on the SMI‐50 list related to endocrine, metabolic, and nutritional disorders and EM were included, as determined by matching with a heading or subheading listed on the MCPEM. Of note, written show notes from podcasts were included in this review.

Exclusion criteria

Each item was screened by a single author (AG, JH, RP, NST) and was excluded if they were deemed irrelevant to endocrine, metabolic, or nutritional disorders or EM. Of note, resources that was better categorized under a different MCPEM category were generally not included. For example, hypovolemia content that primarily focused on ultrasound or septic shock was generally not included because those are classified by MCPEM in their own respective categories. In addition, if a topic was only mentioned in brief, e.g., in a list of the differential diagnosis, it was also generally not included. The authors met multiple times to establish consensus for determining topic relevance. Finally, the following resource types were excluded: posts without text to review, such as pure audio or video links without associated written notes or “show notes”; articles in peer‐reviewed journals; reposts or tagged repositories of posts published initially elsewhere; and non‐English and paid content. Resources were then screened a second time by one author (JH) to ensure the final list of resources met all criteria.

Data extraction and quality and usage assessment

Google Forms (Mountain View, CA) was used as an abstraction and organization tool for the final list of resources and included publication date, described and inferred audience level, subtopic reviewed, author information, and quality assessment via the revised METRIQ (rMETRIQ) score. The rMETRIQ score contains seven categories with scores of 0 to 3 for a maximal score of 21 (Table S2).

To ensure reliable and consistent scoring with the tool, four reviewers (AG, JH, RP, LZ) underwent two rounds of tool rater training. Prior to any training, the average‐measures intraclass correlation coefficient (ICC) was 0.72 (95% confidence interval [CI] = 0.72 to 0.92). After the first round of training, the ICC improved to 0.86 (95% CI = 0.75 to 0.92), and after the second round of training, the final ICC was 0.91 (95% CI = 0.82 to 0.96), indicating a very strong inter‐rater reliability.

A quality cutoff of ≥16 was determined to be high quality based on the modified Angoff method as in the prior SOAR review study. ¹⁴ Like in the previous SOAR edition, each reviewer also determined the appropriate audience level (as indicated by the post or as determined by the rater if not indicated) and appropriate usage (e.g., journal club, postshift reading, or appropriate for on‐shift “just‐in‐time” reference). More than one appropriate usage could be chosen for each article. Finally, prior to evaluations, the authors together reviewed the definitions of each appropriate usage category through discussion and negotiated consensus. ¹⁴

One author (JH) reviewed all posts deemed high quality for duplicate entries and final scores for accuracy. Four of the 124 remaining posts were removed: four duplicates and one toxicologic post. One post met high‐quality criteria after additional points were added due to a missed conflicts‐of‐interest disclosure. Of note, authors did not review any posts for which they authored, edited, or had any other specific conflict of interest.

RESULTS

The search yielded a total of 36,346 resources. Figure 1 displays the search and review results. After removing duplicates (19,413) and journal articles (7108), we were left with 9825 unique resources. Next, four reviewers screened the entries for relevance to the topics as well as to EM. We excluded any remaining journal articles, audio‐only podcasts, image only, non‐English, paid content, nonfunctioning resources, or re‐tagged posts. A remaining count totaled 1181 resources. A secondary screen and full‐text review of these resources yielded 756 final resources that were assessed for quality with the rMETRIQ tool. A total of 121 resources were deemed high quality with an rMETRIQ score ≥16 (see Table 1 for a list of all high‐quality posts).

FIGURE 1.

Search and review results. FOAM, free open‐access medical education resources

TABLE 1.

List of all high‐quality posts (rMETRIQ ≥16)

Topic	Subtopic	Name of First Author	Name of Blog Post	URLE	Level of trainee recommendation	rMETRIQ score
Acid–base disturbances	Metabolic acidosis	Farkas, Josh	PulmCrit: pH‐guided fluid resuscitation & BICAR‐ICU	https://emcrit.org/pulmcrit/bicar‐icu/	PGY‐1 resident, junior, senior, attending	18
Acid–base disturbances	Metabolic acidosis	Farkas, Josh	Is correcting hyperchloremic acidosis beneficial?	https://emcrit.org/pulmcrit/is‐correcting‐hyperchloremic‐acidosis‐beneficial/	PGY‐1 resident, junior, senior, attending	16
Acid–base disturbances	Metabolic acidosis	Rusyniak, Dan	Tox & Hound ‐ aka AKA	https://emcrit.org/toxhound/aka‐aka/	PGY‐1 resident, junior, senior, attending	16
Acid–base disturbances	Metabolic acidosis	Multiple	Alcoholic ketoacidosis ‐ WikEM	https://wikem.org/wiki/Alcoholic_Ketoacidosis	Clerk, PGY‐1 resident, junior, senior	16
Acid–base disturbances	Metabolic acidosis	Spiegel, Rory	CC Nerd‐The Case of the Neutral Documents	https://emcrit.org/emnerd/cc‐nerd‐the‐case‐of‐the‐neutral‐documents/	Clerk, PGY‐1 resident, junior, senior, attending	16
Acid–base disturbances	Respiratory acidosis	Chan, Wendy	VBG vs ABG in Hypercarbia	http://blog.clinicalmonster.com/2016/09/06/vbg‐reliability‐hypercarbia/	Clerk, PGY‐1 resident, junior, senior	18
Acid–base disturbances	Acid–base (mixed)	Farkas, Josh	Fluid selection using pH‐guided resuscitation	https://emcrit.org/pulmcrit/fluid‐selection‐using‐ph‐guided‐resuscitation/	PGY‐1 resident, junior, senior, attending	17
Acid–base disturbances	Acid–base (mixed)	Weingart, Scott	Acid Base in the Critically Ill ‐ Part V ‐ Enough with the Bicarb Already	https://emcrit.org/emcrit/enough‐with‐the‐bicarb‐already/	PGY‐1 resident, junior, senior, attending	17
Adrenal disease	Corticoadrenal insufficiency	Long, Brit	Mimics of Sepsis: What do ED Physicians Need to Know?	http://www.emdocs.net/mimics‐of‐sepsis/	PGY‐1 resident, junior, senior	18
Adrenal disease	Corticoadrenal insufficiency	Farkas, Josh	Adrenal crisis ‐ EMCrit Project	https://emcrit.org/ibcc/adrenal‐crisis/	PGY‐1 resident, junior, senior, attending	17
Adrenal disease	Corticoadrenal insufficiency	Gaillard, Frank	Adrenal hemorrhage | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/adrenal‐haemorrhage?lang=us	PGY‐1 resident, junior, senior, attending	17
Adrenal disease	Cushing's syndrome	Jin, Tee Yu & Gaillard, Frank	Cushing syndrome | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/cushing‐syndrome	Clerk, PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Calcium	Long, Brit	Oncologic Emergencies Part I: Pearls and Pitfalls	http://www.emdocs.net/oncologic‐emergencies‐part‐i‐pearls‐and‐pitfalls/	PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Calcium	Swaminathan, Anand	Hypercalcemia	http://www.emdocs.net/core‐em‐hypercalcemia/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Calcium	Multiple	Hypocalcemia ‐ WikEM	https://wikem.org/wiki/Hypocalcemia	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Calcium	Taliaferro, Dustin	EM@3AM: Hyper‐ and HypoCa	http://www.emdocs.net/em3am‐hyper‐and‐hypoca/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Hypovolemia	Astin, Matt	The SPLIT Trial: Saline vs Plasma‐Lyte Fluid Therapy	https://rebelem.com/the‐split‐trial‐saline‐vs‐plasma‐lyte‐fluid‐therapy/	PGY‐1 resident, junior, senior, attending	18
Fluid and electrolytes	Hypovolemia	Rezaie, Salim	SMART Trial Part 2: Secondary Analysis of Balanced Crystalloids vs Saline in Sepsis	https://rebelem.com/smart‐trial‐part‐2‐secondary‐analysis‐of‐balanced‐crystalloids‐vs‐saline‐in‐sepsis/	Clerk, PGY‐1 resident, junior, senior	18
Fluid and electrolytes	Hypovolemia	Rezaie, Salim	Is the Great Debate Between Balanced vs Unbalanced Crystalloids Finally Over?	https://rebelem.com/great‐debate‐balanced‐vs‐unbalanced‐crystalloids‐finally/	Clerk, PGY‐1 resident, junior, senior	18
Fluid and electrolytes	Hypovolemia	Rezaie, Salim	Does Lactated Ringers (LR) Raise Serum Lactate?	https://rebelem.com/does‐lactated‐ringers‐lr‐raise‐serum‐lactate/	Clerk, PGY‐1 resident, junior, senior	18
Fluid and electrolytes	Hypovolemia	Farkas, Josh	PulmCrit‐ Get SMART: Nine reasons to quit using normal saline for…	https://emcrit.org/pulmcrit/smart/	PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Hypovolemia	Farkas, Josh	IBCC chapter & cast – Fluid selection & pH‐guided fluid resuscitation	https://emcrit.org/pulmcrit/fluid/	PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Hypovolemia	Farkas, Josh	The SPLIT trial: Internal vs. external validity	https://emcrit.org/pulmcrit/the‐split‐trial‐internal‐vs‐external‐validity/	PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Hypovolemia	Rezaie, Salim	Balanced vs Unbalanced Fluids in Pediatric Severe Sepsis	https://rebelem.com/balanced‐vs‐unbalanced‐fluids‐in‐pediatric‐severe‐sepsis/	Clerk, PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Hypovolemia	Levesque, Adrianna & Long, Brit	Fluid Choice in Sepsis: Does it matter?	http://www.emdocs.net/fluid‐choice‐does‐it‐matter/	Clerk, PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Hypovolemia	Kenny, Jon‐Emile	Are balanced crystalloids better than saline? SMART Talk with Dr …	https://pulmccm.org/randomized‐controlled‐trials/smart‐talk‐dr‐michael‐pinksy/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Hypovolemia	Farkas, Josh	PulmCrit‐ Overcoming occult diuretic resistance: Achieving diuresis …	https://emcrit.org/pulmcrit/occult‐diuretic‐resistance/	Attending	16
Fluid and electrolytes	Hypovolemia	Multiple	CT hypoperfusion complex | Radiology Reference Article …	https://radiopaedia.org/articles/ct‐hypoperfusion‐complex?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Hypovolemia	Multiple	Dehydration (peds) ‐ WikEM	https://wikem.org/wiki/Dehydration_(peds)	Clerk, PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Hypovolemia	Rezaie, Salim	The Great Debate Between Balanced and Unbalanced Crystalloids Continues	https://rebelem.com/the‐great‐debate‐between‐balanced‐and‐unbalanced‐crystalloids‐continues/	Junior, senior, attending	16
Fluid and electrolytes	Hypovolemia	Morgenstern, Justin	IV fluid choice part 1: The SPLIT trial	https://first10em.com/split/	Clerk, PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Hypovolemia	Drenzla, Adam	SALT OF THE EARTH (PART 1)	https://intensiveblog.com/salt‐earth‐part‐1/	Clerk, PGY‐1 resident, junior, senior	16
Fluid and electrolytes	Potassium	Meyers, Pendell	Critical Hyperkalemia by H. Pendell Meyers, EMCrit Intern ‐ EMCrit	https://emcrit.org/emcrit/critical‐hyperkalemia/	Junior, senior, attending	17
Fluid and electrolytes	Potassium	Farkas, Josh	Hyperkalemia ‐ EMCrit Project	https://emcrit.org/ibcc/hyperkalemia/	Clerk, PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Potassium	Farkas, Josh	Management of severe hyperkalemia in the post‐Kayexalate era	https://emcrit.org/pulmcrit/management‐of‐severe‐hyperkalemia‐in‐the‐post‐kayexalate‐era/	PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Potassium	Swaminathan, Anand	Is Kayexalate Useful in the Treatment of Hyperkalemia in the …	https://rebelem.com/kayexalate‐useful‐treatment‐hyperkalemia‐emergency‐department/	Clerk, PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Potassium	Singh, Manpreet	Hyperkalemia ‐ The Great Imitator	http://www.emdocs.net/ecg‐pointers‐hyperkalemia‐great‐imitator/	Clerk, PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Potassium	Belcher, Chris & Rogers, Rob	Tumor Lysis Syndrome – Diagnosis and Treatment	http://www.emdocs.net/9077‐2/	Clerk, PGY‐1 resident, junior	17
Fluid and electrolytes	Potassium	Paik, John & Kulstad, Christine	Management of the Sick Dialysis/ESRD Patient	http://www.emdocs.net/management‐of‐the‐sick‐dialysisesrd‐patient/	Clerk, PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Potassium	Meyers, Pendell	A young man with back spasms	http://hqmeded‐ecg.blogspot.com/2018/11/a‐young‐man‐with‐back‐spasms.html	PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Potassium	Helman, Anton	Emergency Management of Hyperkalemia | EM Cases Podcast	https://emergencymedicinecases.com/emergency‐management‐hyperkalemia/	Junior, senior, attending	16
Fluid and electrolytes	Potassium	Long, Brit	Updates in Management of Hyperkalemia	http://www.emdocs.net/emdocs‐cases‐updates‐management‐hyperkalemia/	Clerk, PGY‐1 resident, junior, senior	16
Fluid and electrolytes	Potassium	Weingart, Scott	Treatment of hyperkalemia in the ED	https://emcrit.org/emcrit/hyperkalemia/	Clerk, PGY‐1 resident, junior, senior	16
Fluid and electrolytes	Potassium	Farkas, Josh	PulmCrit‐ BRASH syndrome: Bradycardia, Renal failure, Av blocker …	https://emcrit.org/pulmcrit/brash‐syndrome‐bradycardia‐renal‐failure‐av‐blocker‐shock‐hyperkalemia/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Potassium	Farkas, Josh	Myth‐busting: Lactated Ringers is safe in hyperkalemia, and is …	https://emcrit.org/pulmcrit/myth‐busting‐lactated‐ringers‐is‐safe‐in‐hyperkalemia‐and‐is‐superior‐to‐ns/	Junior, senior, attending	16
Fluid and electrolytes	Potassium	Moussavi, Kayvan & Fitter, Scott	Insulin Dosing in Hyperkalemia	http://www.emdocs.net/insulin‐dosing‐in‐hyperkalemia‐is‐it‐a‐one‐size‐fits‐all/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Potassium	Rezaie, Salim	ECG Changes of Hyperkalemia	https://rebelem.com/ecg‐changes‐hyperkalemia/	Clerk, PGY‐1 resident, junior, senior	16
Fluid and electrolytes	Potassium	Santistevan, Jamie	emDOCs.net – Emergency Medicine EducationSubtle ECG findings…	http://www.emdocs.net/hyperacute‐t‐waves/	Clerk, PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Potassium	Morton, A. Ross	Management of Hyperkalemia with ECG Changes	https://canadiem.org/management‐hyperkalemia‐ecg‐changes/	Clerk, PGY‐1 resident, junior, senior	16
Fluid and electrolytes	Potassium	Awad, Nadia	Thyroid Storm: Treatment Strategies	https://www.aliem.com/thyroid‐storm‐treatment‐strategies/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Potassium	Grock, Andrew	Bicarbonate: Completely Useless?	https://www.aliem.com/bicarbonate‐completely‐useless/	Junior, senior, attending	16
Fluid and electrolytes	Potassium	Hayes, Bryan D	Hyperkalemia Management: Preventing Hypoglycemia From Insulin	https://www.aliem.com/hyperkalemia‐management‐preventing‐hypoglycemia‐from‐insulin/	Senior, attending	17
Fluid and electrolytes	Sodium	Smarandache, Andrei	When and how to treat hyponatremia in the ED	https://canadiem.org/hyponatremia‐in‐the‐ed/	PGY‐1 resident, junior, senior	19
Fluid and electrolytes	Sodium	Farkas, Josh	Hypernatremia & dehydration in the ICU ‐ EMCrit Project	https://emcrit.org/ibcc/hypernatremia/	Senior, attending	18
Fluid and electrolytes	Sodium	Farkas, Josh	Emergent treatment of hyponatremia or elevated ICP with bicarb …	https://emcrit.org/pulmcrit/emergent‐treatment‐of‐hyponatremia‐or‐elevated‐icp‐with‐bicarb‐ampules/	PGY‐1 resident, junior, senior, attending	17
Fluid and electrolytes	Sodium	Farkas, Josh	Taking control of severe hyponatremia with DDAVP	https://emcrit.org/pulmcrit/taking‐control‐of‐severe‐hyponatremia‐with‐ddavp/	Attending	16
Fluid and electrolytes	Sodium	Helman, Anton	Emergency Management of Hyponatremia | EM Cases	https://emergencymedicinecases.com/episode‐60‐emergency‐management‐hyponatremia/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Sodium	Farkas, Josh	Unconventional therapies for hyponatremia: Thinking outside the …	https://emcrit.org/pulmcrit/unconventional‐therapies‐for‐hyponatremia‐thinking‐outside‐the‐collecting‐duct/	PGY‐1 resident, junior, senior, attending	16
Fluid and electrolytes	Sodium	Farkas, Josh	PulmCrit‐ Controlled aquaresis: Management of hypervolemic or euvolemic hyponatremia with oral urea	https://emcrit.org/pulmcrit/aquaresis/	Junior, senior, attending	16
Fluid and electrolytes	Magnesium	Morgenstern, Justin	Torsades de Pointes: Approach to resuscitation	https://first10em.com/torsades‐de‐pointes/	PGY‐1 resident, junior, senior	17
Fluid and electrolytes	Magnesium	Multiple	Hypermagnesemia ‐ WikEM	https://wikem.org/wiki/Hypermagnesemia	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Diabetic ketoacidosis	Lodeserto, Frank	Pediatric DKA: Do Fluids Really Matter? ‐ REBEL EM – Emergency …	https://rebelem.com/pediatric‐dka‐do‐fluids‐really‐matter/	Clerk, PGY‐1 resident, junior, senior	19
Glucose metabolism	Diabetic ketoacidosis	Morgenstern, Justin	IV fluids do not cause cerebral edema in pediatric DKA ‐ First10EM	https://first10em.com/kuppermann‐2018/	Clerk, PGY‐1 resident, junior, senior, attending	19
Glucose metabolism	Diabetic ketoacidosis	Farkas, Josh	Blood gas measurements in DKA: Are we searching for a unicorn?	https://emcrit.org/pulmcrit/blood‐gas‐measurements‐dka‐searching‐unicorn/	PGY‐1 resident, junior, senior, attending	18
Glucose metabolism	Diabetic ketoacidosis	Farkas, Josh	Anatomy of a DKA resuscitation ‐ EMCrit Project	https://emcrit.org/ibcc/dka/	Clerk, PGY‐1 resident, junior, senior	18
Glucose metabolism	Diabetic ketoacidosis	Hughes, Darrel	Is There Any Benefit to an Initial Insulin Bolus in Diabetic…	https://rebelem.com/benefit‐initial‐insulin‐bolus‐diabetic‐ketoacidosis/	Clerk, PGY‐1 resident, junior, senior, attending	18
Glucose metabolism	Diabetic ketoacidosis	Abela, Nikki	Fluid Type and Infusion Rate in Paediatric DKA	https://thesgem.com/2019/05/sgem255‐it‐dont‐matter‐now‐fluid‐type‐and‐infusion‐rate‐in‐paediatric‐dka/	Clerk, PGY‐1 resident, junior, senior, attending	17
Glucose metabolism	Diabetic ketoacidosis	Multiple	Cerebral edema in DKA ‐ WikEM	https://wikem.org/wiki/Cerebral_edema_in_DKA	Clerk, PGY‐1 resident, junior, senior, attending	17
Glucose metabolism	Diabetic ketoacidosis	Lubberdink, Ashley	TREKK Series | Diabetic Ketoacidosis	https://canadiem.org/diabetic‐ketoacidosis/	Clerk, PGY‐1 resident, junior	17
Glucose metabolism	Diabetic ketoacidosis	Farkas, Josh	PulmCrit‐ Dominating the acidosis in DKA	https://emcrit.org/pulmcrit/bicarbonate‐dka/	PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Diabetic ketoacidosis	Farkas, Josh	PulmCrit ‐ Four DKA Pearls	https://emcrit.org/pulmcrit/four‐dka‐pearls/	PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Diabetic ketoacidosis	Reynolds, Catherine, Fisher, Kathrynn & Fairbrother, Hilary	emDOCs.net – Emergency Medicine EducationDiabetic…	http://www.emdocs.net/diabetic‐ketoacidosis‐sneaky‐triggers‐clinical‐pearls/	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Diabetic ketoacidosis	Multiple	Diabetic ketoacidosis | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/diabetic‐ketoacidosis?lang=gb	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Diabetic ketoacidosis	Woods, Jason & Bukowski, Josh	PEM Pearls: Treatment of Pediatric Diabetic Ketoacidosis and the Two‐Bag Method	https://www.aliem.com/pediatric‐diabetic‐ketoacidosis‐two‐bag‐method/	Junior, senior, attending	16
Glucose metabolism	Diabetic ketoacidosis	Monette, Derek	PEM Practice Changing Paper: Clinical Trial of Fluid Infusion Rates for Pediatric DKA	https://www.aliem.com/pem‐practice‐changing‐paper‐clinical‐trial‐fluid‐infusion‐rates‐pediatric‐dka/	Junior, senior, attending	16
Glucose metabolism	Hyperosmolar hyperglycemic state	Multiple	Hyperosmolar hyperglycemic state ‐ WikEM	https://wikem.org/wiki/Hyperosmolar_hyperglycemic_state	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Hyperosmolar hyperglycemic state	Multiple	Hyperosmolar hyperglycemic state | Radiology Reference Article…	https://radiopaedia.org/articles/hyperosmolar‐hyperglycaemic‐state	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Hyperosmolar hyperglycemic state	Multiple	Non‐ketotic hyperglycaemic seizure | Radiology Reference Article…	https://radiopaedia.org/articles/non‐ketotic‐hyperglycaemic‐seizure?lang=gb	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Hypoglycemia	Multiple	Hypoglycemic encephalopathy | Radiology Reference Article…	https://radiopaedia.org/articles/hypoglycaemic‐encephalopathy?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Insulin pump	Hamm, Joel & Aioub, Anasemon	Insulin Pumps: Complications and Emergency Department Presentations	http://www.emdocs.net/insulin‐pumps‐complications‐and‐emergency‐department‐presentations/	PGY‐1 resident, junior, senior, attending	17
Glucose metabolism	Insulin pump	Weingart, Scott	Podcast 198 ‐ Insulin Pumps and Such with Josh Miller, MD	https://emcrit.org/emcrit/insulin‐pumps/	Clerk, PGY‐1 resident, junior, senior, attending	16
Glucose metabolism	Insulin pump	Rezaie, Salim	Diabetic Gastroparesis Needs HUGS	https://rebelem.com/diabetic‐gastroparesis‐needs‐hugs/	PGY‐1 resident, junior, senior, attending	16
Nutritional disorders	Nutritional disorders	Multiple	Osteoporosis | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/osteoporosis‐3?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	16
Nutritional disorders	Nutritional disorders	Long, Brit	Protein Shakes and Dietary Supplements: What are their ingredients and how much is too much?	http://www.emdocs.net/protein‐shakes‐and‐dietary‐supplements‐what‐are‐their‐ingredients‐and‐how‐much‐is‐too‐much/	Senior, attending	16
Nutritional disorders	Vitamin deficiencies	Multiple	Rickets | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/rickets?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	16
Nutritional disorders	Vitamin deficiencies	Groth, Meghan	Mythbusting the Banana Bag	https://www.aliem.com/mythbusting‐banana‐bag/	PGY‐1 resident, junior, senior, attending	16
Nutritional disorders	Wernicke‐Korsakoff	Reyes, Jennifer & Baker, Annalee	Thiamine Deficiency	http://www.emdocs.net/thiamine‐deficiency‐pearls‐pitfalls/	PGY‐1 resident, junior, senior, attending	18
Nutritional disorders	Wernicke‐Korsakoff	Pescatore, Rick	Should You Prescribe Oral Thiamine for Chronic Alcoholics…	https://rebelem.com/should‐you‐prescribe‐oral‐thiamine‐for‐chronic‐alcoholics/	Senior, attending	17
Nutritional disorders	Wernicke‐Korsakoff	Multiple	Wernicke‐Korsakoff syndrome ‐ WikEM	https://wikem.org/wiki/Wernicke‐Korsakoff_syndrome	Clerk, PGY‐1 resident, junior, senior, attending	17
Nutritional disorders	Wernicke‐Korsakoff	Multiple	Wernicke encephalopathy | Radiology Reference Article…	https://radiopaedia.org/articles/wernicke‐encephalopathy	Clerk, PGY‐1 resident, junior, senior, attending	17
Nutritional disorders	Wernicke‐Korsakoff	Swaminathan, Anand	Wernicke's Encephalopathy Archives ‐ REBEL EM – Emergency…	https://rebelem.com/tag/wernickes‐encephalopathy/	Junior, senior, attending	16
Parathyroid disease	Parathyroid disease	Multiple	Parathyroid hormone | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/parathyroid‐hormone	Clerk, PGY‐1 resident, junior, senior, attending	17
Parathyroid disease	Parathyroid disease	Multiple	Parathyroid glands | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/parathyroid‐glands?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	16
Parathyroid disease	Parathyroid disease	Newman, Samantha	PTH versus PTHrP — Small Differences, Big Implications – Clinical…	https://www.clinicalcorrelations.org/2016/03/03/pth‐versus‐pthrp‐small‐differences‐big‐implications/	PGY‐1 resident, junior, senior	16
Pituitary disorders	Pituitary disorders	Multiple	Sheehan syndrome | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/sheehan‐syndrome	Clerk, PGY‐1 resident, junior, senior, attending	17
Pituitary disorders	Pituitary disorders	Long, Drew & Long, Brit	emDOCs.net – Emergency Medicine EducationThunderclap…	http://www.emdocs.net/thunderclap‐headache‐pearls‐pitfalls/	PGY‐1 resident, junior, senior	17
Pituitary disorders	Pituitary disorders	Multiple	Pituitary gland | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/pituitary‐gland?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	17
Pituitary disorders	Pituitary disorders	Multiple	Optic pathway glioma | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/optic‐pathway‐glioma	Clerk, PGY‐1 resident, junior, senior, attending	17
Pituitary disorders	Pituitary disorders	Multiple	Pituitary apoplexy | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/pituitary‐apoplexy?lang=us	PGY‐1 resident, junior, senior, attending	17
Pituitary disorders	Pituitary disorders	Multiple	Empty sella | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/empty‐sella	PGY‐1 resident, junior, senior, attending	16
Pituitary disorders	Pituitary disorders	Multiple	Elevated prolactin (differential) | Radiology Reference Article…	https://radiopaedia.org/articles/elevated‐prolactin‐differential?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	16
Thyroid disorders	Hyperthyroid	Multiple	Thyroid storm ‐ WikEM	https://wikem.org/wiki/Thyroid_storm	PGY‐1 resident, junior, senior, attending	18
Thyroid disorders	Hyperthyroid	Lipp, Chris	CRACKCast E128 – Thyroid and Adrenal Disorders	https://canadiem.org/crackcast‐e128‐thyroid‐adrenal‐disorders/	PGY‐1 resident, junior, senior, attending	18
Thyroid disorders	Hyperthyroid	Multiple	Hyperthyroidism ‐ WikEM	https://wikem.org/wiki/Hyperthyroidism	Clerk, PGY‐1 resident, junior, senior	17
Thyroid disorders	Hyperthyroid	Mackey, Cassandra & Kulstad, Christine	Thyroid Storm and Aortoiliac Occlusive Disease	http://www.emdocs.net/quality‐corner‐thyroid‐storm‐and‐aortoiliac‐occlusive‐disease/	PGY‐1 resident, junior, senior, attending	17
Thyroid disorders	Hyperthyroid	Remus, Kristin & Brenner, Judith	Bedside Rounds Series: Goiter – Clinical Correlations	https://www.clinicalcorrelations.org/2008/02/21/bedside‐rounds‐series‐goiter/	Clerk, PGY‐1 resident, junior, senior	16
Thyroid disorders	Hyperthyroid	Lee, Terrance	Diagnosing hyperthyroidism: Answers to 7 common questions	https://www.aliem.com/2013/10/diagnosing‐hyperthyroidism/	Clerk, PGY‐1 resident, junior, senior	16
Thyroid disorders	Hypothyroid	Fox, Sean	Hypothyroidism and Myxedema Coma — Pediatric EM Morsels	https://pedemmorsels.com/hypothyroidism‐and‐myxedema‐coma/	PGY‐1 resident, junior, senior, attending	17
Thyroid disorders	Hypothyroid	Multiple	Myxedema coma ‐ WikEM	https://wikem.org/wiki/Myxedema_coma	PGY‐1 resident, junior, senior, attending	17
Thyroid disorders	Hypothyroid	Zhao, Leah	Spot the Diagnosis! The case of the Pale Woman	https://canadiem.org/spot‐the‐diagnosis‐the‐case‐of‐the‐pale‐woman/	PGY‐1 resident, junior, senior	17
Tumors of endocrine glands	Tumors of endocrine glands	Multiple	Parathyroid adenoma | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/parathyroid‐adenoma?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	17
Tumors of endocrine glands	Tumors of endocrine glands	Multiple	Central nervous system germinoma | Radiology Reference Article…	https://radiopaedia.org/articles/central‐nervous‐system‐germinoma	PGY‐1 resident, junior, senior, attending	16
Tumors of endocrine glands	Tumors of endocrine glands	Multiple	Craniopharyngioma | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/craniopharyngioma	Junior, senior, attending	16
Tumors of endocrine glands	Tumors of endocrine glands	Multiple	Carcinoid tumor | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/carcinoid‐tumour‐2?lang=us	PGY‐1 resident, junior, senior, attending	16
Tumors of endocrine glands	Adrenal tumor	Multiple	Pheochromocytoma | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/phaeochromocytoma‐1?lang=us	PGY‐1 resident, junior, senior, attending	17
Tumors of endocrine glands	Adrenal tumor	Multiple	Adrenal adenoma | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/adrenal‐adenoma?lang=us	PGY‐1 resident, junior, senior, attending	16
Tumors of endocrine glands	Adrenal tumor	Multiple	Adrenal cortical carcinoma | Radiology Reference Article…	https://radiopaedia.org/articles/adrenal‐cortical‐carcinoma‐1?lang=us	PGY‐1 resident, junior, senior, attending	16
Tumors of endocrine glands	Pituitary tumor	Multiple	Acromegaly | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/acromegaly?lang=us	Clerk, PGY‐1 resident, junior, senior, attending	17
Tumors of endocrine glands	Pituitary tumor	Multiple	Pituitary adenoma | Radiology Reference Article | Radiopaedia.org	https://radiopaedia.org/articles/pituitary‐adenoma	Clerk, PGY‐1 resident, junior, senior, attending	17
Tumors of endocrine glands	Thyroid tumor	Multiple	Medullary thyroid carcinoma | Radiology Reference Article…	https://radiopaedia.org/articles/medullary‐thyroid‐carcinoma‐1?lang=us	Clerk, PGY‐1 resident, junior, senior	17
Tumors of endocrine glands	Thyroid tumor	Multiple	Anaplastic thyroid carcinoma | Radiology Reference Article…	https://radiopaedia.org/articles/anaplastic‐thyroid‐carcinoma?lang=us	PGY‐1 resident, junior, senior, attending	16

Abbreviation: rMETRIQ, revised Medical Education Translational Resources: Impact and Quality.

Topic coverage

All topics and subtopics listed in MCPEM were covered at least once. There was an uneven distribution of FOAM posts for endocrine, metabolic, and nutritional disorders. Nearly half of all resources were categorized under fluid and electrolyte disturbances. The four most represented topics were fluid and electrolytes (42%), followed by glucose metabolism at (16%), then acid–base disturbances (11%), and nutritional disorders (11%). The three most underrepresented topics were parathyroid disease (2%), followed by pituitary disorders (3%) and endocrine tumors (3%). Among subtopics, potassium disorders was the most highly represented, constituting 37% within its own topic category and 15% of all posts (see Figure S1). Table 2 displays the distribution of all topics and subtopics.

TABLE 2.

Endocrine, metabolic, and nutritional disorders SOAR subtopic analysis

Subtopic	Total posts	High‐quality posts rMETRIQ ≥16
Potassium	117 (15%)	20 (17%)
Hypovolemia	51 (7%)	16 (13%)
Diabetic ketoacidosis	67 (9%)	14 (12%)
Sodium	71 (9%)	7 (6%)
Pituitary Disorders/panhypopituitarism	24 (3%)	7 (6%)
Hyperthyroid	34 (4%)	6 (5%)
Metabolic acidosis	40 (5%)	5 (4%)
Wernicke‐Korsakoff	18 (2%)	5 (4%)
Calcium	42 (6%)	4 (3%)
Other tumors of endocrine glands	5 (1%)	4 (3%)
Corticoadrenal insufficiency	28 (4%)	3 (2%)
Parathyroid disease	17 (2%)	3 (2%)
Adrenal tumor	12 (2%)	3 (2%)
Hypothyroid	12 (2%)	3 (2%)
Hyperosmolar hyperglycemic state	8 (1%)	3 (2%)
Insulin pump	3 (0.4%)	3 (2%)
Vitamin deficiencies	35 (5%)	2 (2%)
Nutritional disorders	24 (3%)	2 (2%)
Magnesium	17 (2%)	2 (2%)
Acid–base, mixed	30 (4%)	2 (2%)
Pituitary tumor	5 (1%)	2 (2%)
Thyroid tumor	2 (0.3%)	2 (2%)
Cushing's syndrome	3 (0.4%)	1 (1%)
Respiratory acidosis	5 (0.7%)	1 (1%)
Hypoglycemia	34 (4%)	1 (1%)
Metabolic alkalosis	6 (0.8%)	0 (0%)
Respiratory alkalosis	4 (0.5%)	0 (0%)
Fluid overload	3 (0.4%)	0 (0%)
Phosphorus	17 (2%)	0 (0%)
Hyperglycemia	12 (2%)	0 (0%)
Malabsorption	1 (0.1%)	0 (0%)
Malnutrition	5 (0.7%)	0 (0%)
Thyroiditis	4 (0.5%)	0 (0%)
Total	756	121

Abbreviations: rMETRIQ, revised Medical Education Translational Resources: Impact and Quality; SOAR, systematic online academic resource.

Types of posts

For each of the 756 resources reviewed, each reviewer determined appropriate uses yielding a total of 1964 assigned uses, with an average of 2.6 assigned uses per resource. In descending order of frequency, resources were determined to be appropriate for personalized reading (98%), postshift reading assignment (79%), on‐shift review of best evidence on a topic or just‐in‐time resources (52%), flipped classroom (17%), adjunct to journal club (9%), and individualized interactive instruction credit (7%).

Target audience

Of the 756 posts reviewed, each was reported to be appropriate for at least one type of learner: preclerkship (i.e., first‐ or second‐year medical student), clerkship (i.e., third‐ or fourth‐year medical student), PGY‐1 resident, junior resident, senior resident, and/or attending physician. Authors of the posts specified target audience levels in 167 posts (22% of all posts). Reviewers assigned an appropriate target audience level for the remaining posts. A total of eight posts had only one designated target audience level: seven for attending physicians only and one for preclerkship. All other resources were deemed appropriate for multiple levels (mean = 3.8 levels). In order of frequency, the most common audience was junior resident (91%), PGY‐1 resident (88%), senior resident (81%), clerk (64%), attending (50%), and preclerkship (9%). The distribution among high‐quality resources shifted relatively to an audience with a higher level of training: senior resident (97%), junior resident (95%), PGY‐1 resident (87%), attending (75%), clerk (45%), and preclerkship (0%). Among high‐quality resources, Figure 2 visually describes the distribution of covered subtopics, appropriate level, and the volume of posts.

FIGURE 2.

High‐quality posts (rMETRIQ ≥16) and target audience level by subtopic. rMETRIQ, revised Medical Education Translational Resources: Impact and Quality

Quality assessment

The rMETRIQ scores ranged from 1 to 19 (mean ± SD = 11.4 ± 3.7). Overall, 121 (16.0%) posts met our high‐quality cutoff score of ≥16 (Figure 3). Among these high‐quality posts, the subtopics with the highest numbers of posts were potassium disorders (16%), hypovolemia (13%), and diabetic ketoacidosis (12%; Table 2). Notably, although potassium had nearly twice the number of resources compared to most other topics, only 17% of its resources were deemed high quality compared to hypovolemia at 31%. Similarly, only 10% (7/71) of posts about sodium disorders were deemed high quality. Table 2 shows the distribution of the subtopics and their high‐quality posts.

FIGURE 3.

Distribution of total rMETRIQ scores for all 756 resources reviewed. rMETRIQ scores ≥16 have been demarcated as high quality. rMETRIQ, revised Medical Education Translational Resources: Impact and Quality

DISCUSSION

This study is a continuation of a series to systematically curate FOAM resources by topic and identify high‐quality resources for educators, trainees, and other content creators and users. The challenges of searching and reviewing online literature are not new. ²¹ Frequently, trainees report use of FOAM at the point of care, making it impractical to assess the quality of FOAM in a real‐time clinical setting. ⁵ Gestalt assessment is unreliable, especially in more junior trainees. ¹⁰ , ¹⁹ , ²²

Building on other initiatives like the Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) series, we developed a comprehensive search strategy using two well‐regarded FOAM databases, the SMi and FOAM Search. In particular, websites ranked in the SMi have been shown to correlate with overall quality. ¹⁹ , ²²  Thereafter, all resources underwent an iterative screening process followed by a structured quality assessment with the rMETRIQ scoring tool. The rMETRIQ was a rigorously developed scoring tool that was recently refined based on user feedback. ²³ In the prior SOAR entry, because of resource limitations, only a single reviewer scored resources. In an improvement, for this second entry, four reviewers scored resources and demonstrated strong inter‐rater reliability, verifying that scores are likely consistent.

Topic coverage

As expected, given that this MCPEM category encompasses three somewhat distinct topics—endocrine, metabolic, and nutritional disorders—the resources identified were quite heterogeneous. A total of 33 subtopics were assessed. Every subtopic was covered by at least one post. However, not all subtopics had an identified high‐quality resource, e.g., metabolic alkalosis, respiratory alkalosis, thyroiditis, malnutrition, malabsorption, phosphorus, fluid overload, and hyperglycemia. Resources covering mixed acid–base disorders provide some review of both metabolic alkalosis and respiratory alkalosis. Similarly, hyperglycemia is covered in the context of the subtopics of diabetic ketoacidosis (DKA) and hyperosmolar hyperglycemic state. Posts related to respiratory alkalosis in salicylate poisoning were not included in this review because toxicology is its own MCPEM category. Likewise, posts related to fluid overload in the context of heart failure or renal disease were also excluded.

Among 121 high‐quality posts across 25 subtopics, potassium disorders were most frequently covered (20 resources), followed by hypovolemia and DKA. Most subtopics (18/33) had no more than two high‐quality posts. This is consistent with the findings of a review of all FOAM topics by Stuntz and Clontz, ¹³ which found markedly uneven coverage of topics, further supported by a recent review of 2020 FOAM content by Grock et al. ¹²  Topic coverage seems to favor common critical and/or emergent conditions like DKA and hyperkalemia. Overall, these findings suggest a possible need for further in‐depth and high‐quality coverage of certain subtopics which are considered essential by ABEM standards.

Use of resources

Overall, few resources were deemed by our team suitable for individualized interactive credit (7%) or an adjunct to journal club (7%). Most often resources were recommended for personalized reading (97%), followed by postshift reading assignments (79%). Just over half of all resources were deemed appropriate for on‐shift review or just‐in‐time resources (52%). This is consistent with literature describing trainees reporting frequent use of FOAM at the point of care (POC). Wiki‐based collaborative websites like Radiopaedia and WikEM have been previously reported to be commonly used FOAM resources at the POC. ²⁴ , ²⁵ , ²⁶ Due to the time constraints of on‐shift review, favored POC resources are more likely to be succinct and well structured to increase readability. Surveyed users (70.4%) often found information faster on WikEM compared to other sites. ²⁴ In our review, 80% (115/145) resources from Radiopaedia and WikEM were deemed appropriate for on‐shift review.

Quality of resources

While we found an abundance of resources within this selected topic, only 16% (121/756) met our high‐quality standard. The reasons for heterogeneity of quality as measured by the rMETRIQ score were variable. Of note, for all resources including high‐quality resources, the two rMETRIQ questions scored lowest on average were Q6 (peer review) and Q7 (postpublication commentary; see Table S3). Resources not ranked high quality were more likely to score lower for Q4 (references and citations). Author attributions were at times difficult to discern among many resources. Some sites (e.g., wikis) attributed their work to multiple authors with limited biographical and/or disclosures raising the potential for conflicts of interest and inadequate clarification of training level. For many sites, disclosure information was seldom explicitly presented in the post itself, but available only by navigating other subpages. Additionally, biographical information may be located in a separate templated section of the post or through another link without a time stamp. It was therefore unclear whether the biographical information reflected the rank and title of the author on the date of publication or the current status of the author. No rank or academic affiliation was apparent in 30% (232/756) of all resources. When an affiliation was noted, the academic rank of the author was not apparent in 9% (49/525) of posts.

FOAM sources

Similar to the prior SOAR series topic, genitourinary disorders, we found a high level of contribution from non‐EM specialties contributing to an EM‐relevant FOAM space, notably Radiopaedia (Table S4). The highest number of resources was found on Life in the Fast Lane (15%) and Radiopaedia (12%). The sites with the most high‐quality posts were Radiopaedia (24%), EMCrit Blog (22%), EMdocs (14%), and RebelEM (11%). Overall, this series’ FOAM authors represented a broad range of specialties including but not limited to EM, toxicology, pharmacy, emergency medical services, internal medicine, pediatrics, critical care, orthopedics, nephrology, and radiology. However, within formal EM residency curricula, the frequency of and need for didactic teaching provided by non‐EM specialists are not clear. Rather, ABEM’s MCPEM describes topic categories deemed core content for the practice of EM. Trainees may not receive in‐depth coverage of these specialty topics in their non‐EM clinical rotations. Curating FOAM from non‐EM specialists and incorporating curated high‐quality resources into asynchronous learning opportunities may fill in core knowledge gaps without a substantial additional resource investment from EM training programs.

LIMITATIONS

There were a number of limitations to our protocol and study. First and foremost, the COVID‐19 pandemic resulted in a significant delay between the initial search results and the availability of the authors to review all resources. Within that time frame, there may have been newly published resources and some sites were no longer accessible, i.e., broken links. Second, for pragmatic reasons we excluded audio and video only entries from our review. However, podcast show notes sufficient to stand on their own were again included in our search strategy and included for review. Next, the MCPEM was updated in 2019, after this project began, with the following relevant changes, deletion of “type I [diabetes],” “type II [diabetes],” and “insulin pump malfunction” as subtopics and the addition of “thyroid storm” as a separate subtopic from “hyperthyroidism.” ¹⁶ , ¹⁷

We did not directly assess the accuracy of content included in posts. Assessing accuracy of online educational resources is difficult and no criterion standard exists. The rMETRIQ score does not directly ask for a quality appraisal of the blog post, although the rMETRIQ score includes elements that are likely to correlate with accuracy (e.g., peer review processes, postpublication commentary, references, conflict of interest) and the related METRIQ‐8 score is shown to correlate with expert quality assessments. ¹⁰ , ²⁷ , ²⁸  Next, although we did compute inter‐rater reliability for the rMETRIQ scores, we did not do so for categorization of target audience or usage suitability. Each rater made these determinations independently.

Finally, this is the second iteration of the SOAR process and validity data for this search methodology are yet to be obtained. While a broad search is ideal to prevent missing posts, increasing the search to additional resources would add significant time and resource requirements. As such, the authors concluded that the above search of SMI‐50 and FOAM Search was sufficiently broad while still feasible. It is possible that additional posts were missed by our search strategy.

CONCLUSION

In this second review of the systematic online academic resource series, we present a systematic approach to reviewing online educational resources for the topic endocrine, metabolic, and nutritional disorders and discovered that less than 20% of free open‐access medical education met our high‐quality cutoff. Topic coverage was broad, but with variable quality and disproportionate coverage. Approximately half of subtopics had only three or fewer high‐quality posts. We hope that educators and learners can use this information to find relevant educational resources and that free open‐access medical education content creators can use this information to help identify topics to produce new content.

CONFLICT OF INTEREST

NST receives salary support from the AMA for his role as digital media editor at JAMA Network Open. The other authors have no potential conflicts to disclose.

AUTHOR CONTRIBUTIONS

Jonie J. Hsiao: drafting of the manuscript, study supervision, and analysis and interpretation of the data. Ryan Pedigo: analysis and interpretation of the data. Shirley W. Bae: acquisition of the data. JooYeon Jung: acquisition of the data. Lisa Zhao: analysis and interpretation of the data. Nathan S. Trueger: analysis and interpretation of the data. Teresa M. Chan: statistical expertise and analysis and interpretation of the data. Andrew Grock: study concept and design, study supervision, analysis and interpretation of the data, and critical revision of the manuscript for important intellectual content.

Supporting information

DataSupplement S1

Hsiao JJ, Pedigo R, Bae SW, et al. Systematic online academic resource (SOAR) review: Endocrine, metabolic, and nutritional disorders. AEM Educ Train. 2021;5:e10716. doi: 10.1002/aet2.10716