Keeping up with the literature: A current practice survey and qualitative needs assessment among emergency physicians

Abstract

Purpose

Physicians face the challenge of staying current with a rapidly growing body of evidence and applying it to their practice. How emergency physicians (EPs) do so is unknown. The authors sought to describe and assess needs around EP patterns of evidence‐based medicine (EBM) and continuing medical education (CME) resource use.

Methods

The authors conducted a multicenter, cross‐sectional study in 2019 across 12 tertiary care, community, and suburban emergency department (ED) sites in the greater area of Edmonton. Information on EBM/CME resource use along with barriers and facilitators to staying current was gathered using a rigorously developed survey tool, distributed electronically and by mail. Responses were tabulated and subgroups analyzed using MANOVA and ANOVA tests. Thematic analysis of comments used a phenomenological lens.

Results

A total of 118 EPs (40.1%) completed the survey. Listening to podcasts, attending EM conferences, and subscription‐based resources were preferred for staying current. Resource use differed by years in practice but not by age, sex, training background, or site type. EBM had an important impact on respondents’ practice (average rating 3.8 out of 5, with 5 indicating “practice changing”). Time was an important barrier. Most (62.7%) felt that they did not spend enough time, despite spending a median of 4 to 5 h monthly on EBM. Facilitators (including journal club summaries or lists of practice‐relevant papers) had only moderate impacts. Thematic analysis found three themes (importance of EBM, implementation challenges, and dissemination of EBM) and 13 subthemes.

Conclusion

EPs preferentially chose podcasts, conferences, and subscription‐based resources to stay current with EBM; time was the biggest barrier. These findings help ED leads and educators tailor CME to physician learning preferences to maximize application of EBM to clinical practice. The next steps include developing/curating resources and disseminating the survey on a larger scale to identify opportunities for shared virtual resources.

INTRODUCTION

Modern clinical practice is guided by a rapidly proliferating body of literature. ¹  The introduction of electronic medical journals and the advent of free open access medical education (FOAM) resources has improved availability and access to evidence‐based medicine (EBM) resources, including primary literature, evidence‐based reviews, and clinical practice guidelines. ² , ³ Emergency physicians (EPs) face the challenge of staying current with this broad and rapidly evolving body of literature.

In this era of online learning, emergency clinicians have many avenues to access and stay current with the latest evidence—yet how and to what extent they do so has not been described in detail. The objectives of this study are to determine the types of resources emergency department (ED) physicians use for continuing medical education (CME) and to conduct a needs assessment to determine what resources might help physicians stay current with EM literature.

METHODS

Research setting and population

The Edmonton zone includes a network of 12 EDs that service a population of over 1.4 million patients in the greater Edmonton area of Alberta, Canada. The Edmonton zone is the second largest zone in the province and sees over 2 million annual ED visits. The zone includes three academic teaching hospitals, three community hospitals, and six suburban (peripheral) hospitals in the greater Edmonton area. We identified and surveyed all EPs practicing in Edmonton zone EDs through a current and complete list of email addresses provided by the deputy zone clinical department head.

Survey development

A descriptive cross‐sectional survey was developed following Burns methodology ⁴ and in accordance with The Survey Checklist for survey development. ⁵ Briefly, items (i.e., questions were generated through a literature review and author input). Next, an expert panel (five practicing EPs with expertise in emergency medicine, CME resource development, and EM administration) provided feedback on each question. Items were modified or removed accordingly. The revised survey was distributed for pilot testing and clinical sensibility testing (using Burns’ clinical sensibility tool ⁴ ) to five EPs who trained in the Edmonton zone but currently practice elsewhere in Canada. The survey was then further revised, forming the final version (Appendix S1).

The survey consists of three sections: demographics, current practice patterns, and attitudes toward EBM/CME. Demographic data include age, sex, formal training, years in practice, and ED site(s) of practice. Formal training includes Fellow of the Royal College of Physician and Surgeons (FRCPC) specialty training in EM (5 years) or pediatrics (4 years), pediatric EM fellowship (2 years), or Canadian College of Family Physicians (CCFP) training in family medicine (2 years) or CCFP‐EM (family medicine with an 1 additional year of specialized EM training). Current practice captures the amount of time spent reading EM literature, current sources of CME, and ideal time spent on EBM. To elicit current attitudes and thoughts around EBM, we ask questions on the importance of staying current with literature and the importance of EBM to practice. Response options include 5‐point Likert scale, ranking, and multiple‐choice and free‐text questions.

Survey distribution

We used a modified Dillman method for survey distribution. ⁶ , ⁷ In September 2019, all EPs in the zone were emailed a personalized letter of introduction, with the survey emailed 7 days later. Reminders were sent on Days 14 and 21. Study data were collected and managed using REDCap electronic data capture tools hosted at the University of Alberta. ⁸ REDCap randomly assigned a personalized number to each participant (ensuring anonymity), while tracking those who had not completed the survey. EPs who did not respond electronically received a paper copy of the survey on Day 35 in work mailboxes that EPs check regularly, with instructions to submit completed surveys into an anonymous drop box placed visibly nearby. Drop boxes were collected three weeks later and transcribed by a data analyst (AG).

Data analysis

Quantitative responses were tabulated and displayed numerically, using percentages or means where appropriate. With the possibility that learning needs differ by demographic factors, a priori subgroup analyses stratified EBM/CME resources and needs assessment feedback by ED site type (see below), years in practice (<5, 5–9, 10–14, 15–19, and ≥20 years), age (≤35, 36–45, 46–55, 56–65, or ≥66 years of age), and sex (male, female, other/unspecified). Post hoc, we also added training background (specialty training in EM or pediatric EM vs. other). Four individuals not indicating a site type were dropped from subgroup analyses. Use of a given resource was categorized as use of any use versus nonuse. Physicians who worked at more than one site were categorized according to the most specialized site (i.e., EPs working at peripheral and academic sites were classified as academic, peripheral and community were classified as community) because they have access to the CME resources provided by more academic sites. Barriers and facilitators to access of EBM/CME resources were categorized as minor (<2.5 out of 5), moderate (2.5–3.5 out of 5), or important (>3.5 out of 5).

Because we had multiple outcome variables of interest, we entered data into a MANOVA model in which EBM resource utilized, barriers, and facilitators were included as the dependent variables; sex, years in practice, age, site of work, and training background were the independent variables. We utilized Pillai's trace (α = 0.05) due to its robustness. For the independent variables that demonstrated a significant difference, we then used one‐way ANOVA (α=0.05) to test for subgroup differences in each outcome. Statistical analysis was conducted using Stata (StataCorp).

Qualitative feedback was summarized using a thematic analysis of free‐text responses, grounded in phenomenology, with the goal of understanding the lived experience and needs of respondents. This approach was selected because of compatibility with our specific research questions and the survey format. Thematic analysis of deidentified comments was conducted by two investigators with experience in thematic analysis: an EM resident (ICG) and EM staff physician (BAS). Neither investigator has authority over participants, and it was not possible to individually identify any of the deidentified comments. Investigator triangulation was used to enhance trustworthiness of the qualitative analysis, ⁹ as this was feasible with deidentified survey data. Qualitative reporting was conducted in compliance with the standards for qualitative research reporting. ¹⁰

Ethics

Each participant was provided information on informed consent. Consent could be withdrawn up to and including the point of survey submission. This study was approved by the University of Alberta Research Ethics Board (Pro00079106).

RESULTS

We received 118 responses from a total of 294 surveys administered (40.1% response rate). Of the 118 surveys, 99 (83.9%) were captured electronically and the remainder via subsequent paper survey. Respondents represented a diversity of practicing physicians based on age, years in practice, and site (Table 1). While there were proportionately more males (61.0%), respondents were evenly spread across years in practice.

TABLE 1.

Characteristics of respondents

Characteristic	n	% (N = 118)
Age (years)
≤35	27	22.9%
36–45	41	34.7%
46–55	33	28.0%
56–65	11	9.3%
≥66	1	0.8%
No response	5	4.2%
Sex
Female	33	28.0%
Male	72	61.0%
Other	2	1.7%
Prefer not to say	6	5.1%
No response	5	4.2%
Years in practice
<5	28	23.7%
5–9	25	21.2%
10–14	21	17.8%
15–19	16	13.6%
>20	23	19.5%
No response	5	4.2%
Training background
FRCPC in EM	41	34.7%
CCFP‐EM	48	40.7%
CCFP (no official EM training program)	10	8.5%
General pediatrics (FRCPC)	6	5.1%
PEM fellowship	11	9.3%
Other	5	4.2%
No response	6	5.1%
Site type(s) of work
Tertiary care hospital	74	62.7%
Community hospital	39	33.1%
Suburban hospital	40	33.9%
No response	4	3.4%
Number of sites worked
1	72	60.5%
2	30	25.2%
3	11	9.2%
≥3	12	10.1%

Abbreviations: CCFP, Canadian College of Family Physicians; FRCPC, Fellow of the Royal College of Physicians and Surgeons of Canada; PEM, pediatric emergency medicine.

Practice patterns

Physicians ranked listening to podcasts, attending EM conferences, and subscription‐based resources as their top ways of staying up to date on evidence (Table 2, Appendix S2). Ninety‐one physicians (77.1%) listened to podcasts in the past year, with a mean of 2.4 different podcasts per physician. The five most popular podcasts were EM:RAP (Emergency Medicine Reviews And Perspectives, 57.6%), EMCrit (Emergency Medicine and Critical Care 32.3%), EM Cases (29.7%), ALiEM (Academic Life in Emergency Medicine, 20.3%), and the SGEM (Skeptic's Guide to Emergency Medicine, 19.5%; Appendix S2).

TABLE 2.

Percentage of EM physicians ranking the resource in their top three CME resources

Resource	Overall	Tertiary care	Community	Suburban
Medical journals	12.0	10.6	19.5	13.4
Attending emergency medicine conferences	17.5	17.1	17.1	17.9
Attending emergency medicine grand rounds	7.5	8.3	9.8	3.0
Listening to podcasts	18.7	19.0	17.1	17.9
Reading blogs	6.9	7.4	2.4	9.0
Reading society guidelines or position statements (on society website or in a publication)	10.2	11.6	7.3	9.0
Subscription‐based resources	17.5	17.6	22.0	13.4
Recent edition of EM textbooks (e.g., Rosen's, Tintinalli)	4.5	4.2	0.0	9.0

Abbreviation: EM, emergency medicine.

Ninety‐seven physicians (82.2%) attended conferences in the past 2 years, averaging 1.6 different conferences. The five most attended conferences were Edmonton Emergency Physicians Association Conference (37.3%), Canadian Association of Emergency Physicians (30.5%), Critical Care in the Emergency Room (16.9%), American College of Emergency Physicians (11.0%), and Essentials of Emergency Medicine (11.0%).

Nearly all physicians (n = 111, 94.1%) accessed subscription‐based clinical reference tools in the past year (Appendix S2). The most commonly used resource was UpToDate (77.1%), trailed by Medscape (32.2%) and Lexicomp (21.2%). The least commonly used resources were social media resources, used by 50% of physicians; among these, Twitter (20.3%) was most popular.

Other ways of staying current included reading journals (90.7%, most commonly Canadian Medical Association Journal, Canadian Journal of Emergency Medicine, Annals of Emergency Medicine, and New England Journal of Medicine), published guidelines or position statements (58.5%), attending local departmental education programs (54.2%), attending grand rounds (45.8%), and reading blogs (65.3%; most commonly Life in the Fast Lane, ALiEM, CanadiEM, and the SGEM). Forty‐eight (40.7%) physicians used social media to stay current, including Twitter (20.3%) and Facebook (16.9%). Responses are further detailed in Appendix S2.

EPs spent a median of 4 to 5 h every month on EBM or CME. Roughly one in three physicians (35.6%) felt they spent the right amount of time on this topic, whereas most (62.7%) felt they spent too little.

Needs assessment

Respondents rated the impact of potential barriers and facilitators to staying current with the literature using a 5‐point scale (higher values indicate larger impact; Appendix S3). EPs felt EBM had an important impact on their practice (average 3.8).

The biggest barrier was time (mean rating 4.0). Lack of user‐friendly resources, feeling unprepared to critically appraise the literature, and being unsure of the impact on one's practice had moderate impact. Most (66.1%) indicated that lack of interest was not a barrier.

Several potential facilitators to staying current with EM literature, research, and/or EBM arose, all of which had moderate impacts, including one‐page summaries of papers discussed at local journal clubs, periodic lists of major papers of interest (with links), and audio summaries of journal clubs/papers (means of 3.4, 3.4, and 3.3, respectively). Most physicians (96.0%) preferred emailed journal club summaries over in‐ED posters. While cost ranked lower among potential barriers, physicians rated accessibility to resources free of charge as a moderate impact facilitator.

Subgroup analyses

Resources used

Use of CME resources differed by years in practice (p ≤ 0.001) but not by any other subgroup. Years in practice had a significant effect on participating in a journal club (p = 0.01), using free‐access websites (p = 0.03), using subscription‐based references (p = 0.02), reading blogs (p = 0.04), and using social media for education (p = 0.02).

There was no significant difference in preferred (top three) resources for staying up to date, by years in practice (Appendix S2); however, there was a trend toward preferring journals as years in practice increased, and preferring podcasts as years in practice decreased. Interestingly, as years in practice increased, the number of blogs and podcasts used decreased.

The time spent on EBM differed only by years in practice (p = 0.03), with less time spent on EBM after 15 years in practice. The impact of EBM on clinical practice did not differ within any subgroup.

A post hoc sensitivity analyses substituted specific site type (for most specialized site type) and total number of resources used within each CME category (for use/nonuse of that resource type). There were no significant differences.

Facilitators and barriers

Facilitators to staying current with EBM did not differ in any subgroup. Barriers to staying current with EM literature differed only by most specialized site type (p = 0.04). “Unsure if it really makes an important difference in your practice” was the only barrier that differed between sites (p = 0.01), which was driven by a difference between tertiary care and community sites.

Thematic analysis of comments

Thematic analysis encompassed 91 separate free‐text comments on facilitators, barriers, and general feedback. We found three main themes (importance of EBM, implementation challenges, and dissemination of EBM) and 13 subthemes (Table 3).

TABLE 3.

Thematic analysis of free text comments

Theme	Subtheme	Exemplary quotes
Importance of EBM	Staying current with evidence	“Always new things evolving.” (site type 3) “Huge breadth of practice requires staying up to date—ever‐growing body of evidence (particularly PEM) results in and necessitates change to practice.” (site type 1)
	Self‐improvement	“Medicine changes daily; want to give best care possible to my patients.” (site type 1) “It is always important to examine one's practice, and wonder if one is up to date.” (site type 1)
	Patient care/quality care	“I feel that staying current in my practice is a very important element of good clinical care.” (site type 1) “Medicine changes daily; want to give best care possible to my patients.” (site type 1)
Implementation challenges	Evolution	“Constantly evolving practice and research is difficult to keep up with without these resources.” (site type 1) “I find it hard as a (somewhat) new attending to keep up with everything. […] it is incredibly hard to keep up to date on most things.” (Site type 1)
	Access	“Unsure which resource is ‘best bang for your buck.’” (site type 1) “Lack of EM conferences and workshops in the province. Most of the invitation I receive are in Ontario.” (site type 3) “Ease of access at point of care.” (site type 1)
	Individual challenges	“It is important, but finding time for CME can be a challenge.” (site type 3) “Finding the motivation and where to start since EM is basically everything.” (site type 1) “As a new graduate I more so just want a break from keeping up with EBM but I'm sure in a year when I don't feel so overwhelmed with learning day to day on shift I will seek out more EBM.” (site type 1) “All good stuff, but I am near end of practice and less interested than I was in earlier life.” (site type 2)
	Practical application	“It is tough to be an early adopter as the most up‐and‐coming research is sometimes not considered acceptable yet or ready for mainstream uptake.” (site type 1) “A large part of my practice is quite routine day to day common ED problems and is not impacted much by cutting edge new information.” (site type 1) “I need strong evidence for full practice to change and for specialty to as a group move towards a change in practice and will not change practice just because one or two small studies suggests something.” (site type 1) “EBM is a tool and a guide in my view. It may not always apply and when applied learners must learn the details of the study to understand to whom it applies and most importantly when it is NOT applicable. I find EBM can at times be misapplied, which makes it at times dangerous or leads to unnecessary investigations.” (site type 1)
	Institutional inertia	“There is significant institutional impedance for implementing change in my department. We cannot meet standard of care for things that are universally agreed upon let alone for new developments. Administration obstructs all attempts at improvement.” (site type 2) “I find that lots of new evidence in the literature is not embraced by local practice culture or with our consultants’ practice.” (site type 1)
	Change management	“Things change slowly, and implementation at a department local level is what really changes things, not on an individual level … to much variance to make a difference on a population.” (site type 1) “The physicians who are horribly outdated and actually need it will not access it. This comes from my previous experience in leadership roles that I've held within our specialty.” (site type 1)
Dissemination of EBM	Dissemination	“Email resources/access will be great.” (site type 3) “‘Bottom line’ practice changing summaries of peer‐reviewed literature.” (site type 1) “Knowing how to best access and use an RSS feed would be awesome, especially if you gave me a list of sites to attach to it.” (site type 1)
	Applicability and availability	“SUCCINCT summaries with local influence considered.” (site type 1) “I always wish that I had more time to review more literature and keep up to date.” (site type 3) “Would love the chance to attend zone wide journal clubs, rounds, etc. Would really help unify care in the zone.” (site type 3)
	Guidance with EBM	“Curated list by our emergency physician group or residency program.” (site type 1) “Simply suggesting popular podcasts and other CME for emergency medicine is helpful as those of us not FRCPC or CCFP‐EM may not be as familiar with available resources.” (site type 3)
	Implementation	“If there is a zone effort to provide EBM, it should be targeted for a purpose to improve patient care so it should be introduced with a QI initiative. Just putting info out for the sake of knowledge, that's great but won't make the ‘system’ better.” (site type 1) “Knowing I could actually implement the changes in my department would be beneficial […].” (site type 2)

Abbreviations: CME, continuing medical education; EBM, evidence‐based medicine.

Physicians highlighted the important role EBM plays in personal and professional improvement. Several expressed that practicing EBM was synonymous with quality and excellence in patient care.

Respondents noted several challenges to EBM uptake and implementation, including difficulty keeping up with the rapid growth and volume of literature, feeling overwhelmed by the many resources to choose from, and difficulty knowing where to start (given the breadth of topics relevant to EM). Some had difficulty finding time or were fatigued or disinterested. Others highlighted a paucity of local/regional CME resources. Multiple physicians identified challenges with translating evidence into practice, including the strength (or weakness) of evidence, its interpretation, and its applicability. Some felt more leadership at the departmental level was required—along with agreement among specialist colleagues—to incite change (including among physicians less interested in EBM).

EPs widely agreed that resources should be succinct, practice‐focused, and judiciously shared—ideally electronically. Many physicians desired locally based resources (such as practice updates, journal clubs, or resident teaching) and guides to existing resources (including FOAM resources). Some highlighted opportunities for departmental‐level support to create evidence‐based changes (for example, tying EBM into quality improvement initiatives).

DISCUSSION

This is the first study to report on both traditional and new (FOAM) resources EPs use for CME: predominantly conferences, subscription‐based resources and podcasts. As online resources expand, patterns of CME use observed over a decade ago (on‐the‐job contact with other physicians, formal ED‐based teaching, and medical journals) differ substantially from our findings. ¹¹ Similar to findings elsewhere in Canada, ¹² we report that podcasts are the most popular FOAM resource. In keeping with previous studies, social media (e.g., Twitter, Facebook) were less popular avenues for obtaining CME. ¹³

Although physicians value using EBM in their practice, time is the most important barrier to staying current and many are overwhelmed navigating the volume of literature and FOAM resources. Elsewhere in Canada, EPs feel similarly overwhelmed. ¹²  Most physicians felt they spent too little time on EBM and attributed this to shift work, multiple commitments, and fatigue. This is in keeping with other reports on attitudes toward EBM among EPs in Western countries ¹⁴ and across physician specialties ¹⁵ , ¹⁶ , ¹⁷ ). Lack of interest was not a barrier for most EPs.

Interestingly, our analyses suggest that the era during which a physician trains (rather than their age) forms patterns for staying current. This has implications for residency training programs (how residents learn to integrate EBM into practice) and ED leadership (addressing variations in practice within a heterogenous physician group).

Our findings highlight opportunities for CME development. An easy starting point for educators may be to create guides to navigating resources and succinct, practice‐focused evidence updates. Combining grassroots and top‐down approaches can make a positive impact. ¹⁸ Department leaders can support physicians wishing to introduce evidence‐based initiatives, including quality improvement research around evidence‐based practice as well as integrating evidence into routine patient care (e.g., protocols, evidence‐based alerts in electronic records, departmental memos regarding best practices, and practices endorsed by local specialist consensus). Health regions can develop educational modules, memos, and conferences around the latest evidence. Resident training programs can participate in developing local educational resources, such as concise summaries from journal clubs or grand rounds. As socially distant learning channels grow, so do opportunities to tap into CME resources from other jurisdictions.

LIMITATIONS

Our study is not without limitations. Though we could not measure how closely our sample reflects all EPs in the zone, participants represented all demographic categories and EDs. We could not identify the resources/supports available at each hospital for CME, which may impact resources used. Our survey did not capture frequency of use: a physician indicating they read a journal could have read a single article or the entire journal. Sensitivity analyses using the number of journals read showed no significant differences. Our qualitative analysis revealed several themes, yet deeper understanding was limited by use of a structured survey. Further studies using different qualitative methods, such as focus groups, may better describe physicians’ lived experiences.

CONCLUSIONS

Here we present a rigorously developed survey and its responses, highlighting practice patterns as well as barriers and facilitators around how emergency physicians stay current with literature in their broad field of practice. Podcasts, emergency medicine conferences, and subscription‐based resources are the main modalities, with years in practice driving subgroup differences.

Although most physicians value evidence‐based medicine, they highlighted individual, departmental, and regional barriers to integrating evidence‐based medicine into practice. Residency programs and ED leadership can use our findings to develop continuing medical education resources. Expanding this survey to further understand trends can make continuing medical education more impactful, ultimately improving patient care.

CONFLICT OF INTEREST

All authors report no conflict of interest.

AUTHOR CONTRIBUTIONS

Isabelle N. Colmers‐Gray, David J. Ha, and Brandy A. Stauffer conceptualized the project. Isabelle N. Colmers‐Gray developed the initial survey tool, with input from Brandy A. Stauffer and David J. Ha. Isabelle N. Colmers‐Gray, David J. Ha, and Brandy A. Stauffer participated in collecting physical surveys. Aliyah Gauri developed the electronic survey tool and inputted data from paper surveys. Isabelle N. Colmers‐Gray and Leandro Solis Aguilar conducted quantitative analyses while Isabelle N. Colmers‐Gray and Brandy A. Stauffer conducted the qualitative analysis. Isabelle N. Colmers‐Gray wrote the first draft of the manuscript and all authors had input on subsequent versions.

Supporting information

Appendix S1

Appendix S2

Appendix S3

Colmers‐Gray IN, Solis Aguilar L, Gauri A, Ha DJ, Stauffer BA. Keeping up with the literature: A current practice survey and qualitative needs assessment among emergency physicians. AEM Educ Train. 2022;6:e10761. doi: 10.1002/aet2.10761