Maintaining procedural skills for academic emergency medicine faculty: A needs assessment

Abstract

Background

Emergency physicians require competence performing critical and routine procedures. The clinical practice of emergency medicine (EM) alone may be insufficient for the acquisition and maintenance of skills. Prior studies suggest the presence of trainees in academic settings and/or the low frequency of procedures increase the risk of skills attrition among faculty. We sought to develop a valid needs assessment survey to inform a faculty procedural skills (FPS) maintenance curriculum.

Methods

A Web‐based FPS survey was designed to assess experiences performing procedures, self‐reported confidence with procedures, and learning preferences for skills maintenance. The survey was administered at a large academic department of EM. Responses were analyzed to determine survey construct validity, faculty attitudes about procedural attrition, and preferred learning methods.

Results

Among EM faculty, confidence was significantly higher for common versus uncommon procedures (p < 0.001). EM faculty respondents reported significantly greater confidence than pediatric EM (PEM) faculty for both common adult procedures (EM mean = 3.7 [±0.3], PEM = 3.0 [±0.4], p < 0.001), and uncommon adult procedures (EM = 2.7 [±0.4], PEM = 2.1 [±0.5], p < 0.001). PEM faculty reported significantly greater confidence with pediatric procedures than EM faculty (PEM mean [±SD] = 3.5 [±0.8], EM = 2.2 [±0.8], p < 0.001). Nearly all faculty (93% [52/56]) agreed that procedural attrition is a concerning problem, and 80% (44/56) had personally experienced it. The most preferred learning methods were task trainers and simulation. Faculty preferred learning environments with faculty peers (91%) over mixed groups with trainees (50%).

Conclusions

Significant differences in procedural skills confidence between common and uncommon procedures, and between EM and PEM faculty, indicate that the FPS survey displayed appropriate construct validity. The finding that skills attrition is prevalent among EM and PEM faculty highlights the need for skill maintenance programming, preferably in peer groups employing task trainers and simulation.

INTRODUCTION

Emergency physicians (EPs) must be competent to perform critical and routine procedures. Prior to independent clinical practice, trainees must demonstrate competency by performing a minimum number of essential procedures as defined by the Accreditation Council on Graduate Medical Education (ACGME). ¹ , ² Most of these occur in the course of patient care; however, some rare procedures (e.g., cricothyrotomy) are learned through skills labs, simulation, or other methods. Formal assessment of procedural skills beyond training is uncommon, despite evidence that technical proficiency depends on sustained deliberate practice. ³ , ⁴ , ⁵ There are no requirements for practicing, board‐certified EPs to demonstrate ongoing procedural competency. ⁶ Prior studies suggest that clinical practice alone is insufficient to maintain procedural skills. ⁷ , ⁸ , ⁹ , ¹⁰ Academic faculty who supervise trainees encounter fewer opportunities to directly perform procedures, potentially placing them at higher risk for skills attrition. ⁸ , ⁹ Studies suggest that physicians have limited ability to self‐assess for procedural competence. ¹¹ , ¹² , ¹³ It is unclear how willing academic faculty are to admit a lack of confidence performing procedures for which they are credentialed. ¹⁴

Educational programs to ensure maintenance of skills are prevalent. Examples include continuing medical education refresher courses for high‐risk, low‐frequency procedures (e.g., cricothyrotomy) and programs focused on procedures providers had limited experience with during training (e.g., video laryngoscopy). Some departments or institutions may mandate focused retraining after an adverse outcome or in response to procedural complications. Despite various programs and formats, there is a paucity of literature on emergency medicine (EM) faculty preferences for procedural maintenance education.

We sought to design a recurring procedural skills maintenance program for EM and pediatric EM (PEM) faculty in a large academic department, employing an established model for curriculum development. ¹⁵ As a first step, we developed the faculty procedural skills (FPS) needs assessment survey. The specific aims of this study were to: (1) assess the survey's construct validity, (2) determine faculty experiences with procedural skill attrition, and (3) identify ongoing training needs and faculty learning preferences.

METHODS

Study design

The FPS needs assessment survey was a cross‐sectional study of EM and PEM faculty at a single, large academic EM department.

Survey instrument

The FPS survey was developed iteratively, based on a comprehensive list of EM procedures from key sources including the ACGME program requirements for EM and PEM, ¹ , ¹⁶ the 2019 Model of the Clinical Practice of Emergency Medicine, ¹⁷ previously published studies, ⁷ , ⁸ and hospital credentialing forms from the participating department. Overlapping procedures were identified and consolidated. The final list was categorized into three groups: pediatric‐specific (e.g., pediatric intubation), common adult (e.g., adult CPR), and uncommon adult (e.g., thoracentesis). The survey was pilot tested by EM fellows who were ineligible to participate in the study and subsequently revised for content, cognitive flow, logic, timing, and functionality. The final survey consisted of three sections: (1) respondent characteristics and experiences performing and supervising a range of EM procedures, (2) degree of confidence performing specific procedures, and (3) attitudes about skill attrition and learning preferences for skill maintenance. The survey instrument can be found in Data Supplement S1 (available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10648/full).

Study setting and population

Study subjects were EM faculty who provide clinical care and supervision of trainees at one or more EDs in our hospital system. All subjects were board certified or board eligible in EM or PEM. EM faculty in our system may encounter a small percentage of pediatric patients; however, their practice almost exclusively involves adult patients. Study authors who developed the survey were excluded. Subjects were recruited via email with an informational script and an invitation to complete the survey online using REDCap (Research Electronic Data Capture), a secure, Web‐based software platform designed to support data capture for research studies (https://projectredcap.org/). Participation was anonymous, voluntary, and uncompensated. Those completing the survey qualified to win a $50 gift card. Data were deidentified and stored using REDCap. The project was reviewed by the institutional review board and determined exempt.

Outcome measures

The items of primary interest related to experiences with skill attrition, ongoing training needs, and faculty preferences for learning methods and environments. Other measures included overall response rate and respondent characteristics such as years of experience, board certification, and the estimated percentage of shifts during which respondents supervised trainees.

Data analysis

To assess for construct validity, responses were analyzed for differences in reported confidence between general EM and PEM faculty as they related to common and uncommon adult‐specific procedures and pediatric‐specific procedures. These items were graded on a 1 to 4 scale, with higher values representing greater confidence. Mean confidence across the items was calculated (maximum = 4). A series of t‐tests were conducted to assess differences in general EM and PEM confidence. Faculty responses to items about skill attrition and learning preferences were then analyzed. These items were graded on a 5‐point scale (strongly disagree to strongly agree), with higher values representing greater agreement. Percentage agreement (with 95% confidence interval [CI]) was calculated by combining “strongly disagree” with “disagree” and “strongly agree” with “agree” responses for each survey item. Because preliminary analyses revealed no significant differences in the responses of EM and PEM faculty to these items, they were combined to provide an overall analysis. The survey instrument is available as a supplement.

RESULTS

A total of 56 faculty completed the FPS survey out of 99 eligible (57% response rate). Of those, 40 were board‐certified in EM only and 16 in PEM only. There was no significant difference in years of clinical experience (p = 0.56), but significantly more PEM faculty (75%, 95% CI = ±11%) than EM faculty (6%, 95% CI = ±7%) reported supervising trainees on more than 75% of their shifts.

Construct validity of FPS was assessed by examining the confidence items scales and predicted difference in scores between PEM and EM faculty. The PEM faculty had significantly higher mean confidence with PEM‐specific procedures (p < 0.001), and EM faculty had significantly higher confidence with both common and uncommon adult‐focused procedures (p < 0.001). Among EM faculty only (n = 40), there was significantly greater mean confidence with common procedural skills compared to uncommon ones (mean Δ = 0.97, 95% CI = ±1.0).

Table 1 summarizes faculty experiences with and attitudes about procedural skills attrition. All respondents agreed or strongly agreed that ongoing procedural skills training is important, and the majority agreed or strongly agreed that skills attrition was a concerning problem (96%, 95% CI = ±5.1%); that attrition was more likely in academic settings (94.5%, 95% CI = ±6%); that faculty should be required to demonstrate procedural competence (83%, 95% CI = ±8.2%); and that faculty should have opportunities to retrain for uncommon procedural skills (94.5%, 95% CI = ±3.2%). The majority of respondents (78%, 95% CI = ±11%) agreed that they had personally experienced procedural skill attrition.

TABLE 1.

Faculty experiences with and attitudes about procedural skills attrition

Item	Disagree/strongly disagree	Neutral	Agree/strongly agree
Maintenance of procedural skills is important	0	0	56 (100)
Procedural skill attrition is a concerning problem	1 (1.5)	3 (5.5)	42(96)
Procedural skill attrition is more likely to occur among faculty in teaching environments where trainees perform the majority of procedures.	0	3 (5.5)	53 (94.5)
Procedural skill attrition among EPs is an important patient safety issue	0	6 (11)	50 (89)
Faculty should be able to demonstrate competence for procedures they are credentialed	2 (3)	8 (14)	46 (83)
Faculty should have regular opportunities to retrain for rarely performed procedures	1 (1.5)	0	55 (98.5)
I have observed procedural skill attrition	6 (11)	10 (18)	40 (71)
I have personally experienced procedural skill attrition	6 (11)	6 (11)	44 (78)
I am interested in participating in faculty development	2 (3)	3 (5.5)	51 (91.5)

Data are reported as n (%).

Simulation training (94.5%, 95% CI = ±10.5%) and task trainer models (86%, 95% CI: ±8.5%) were the most preferred learning methods. Most respondents agreed that training with faculty peers provided the most comfortable learning environment (91%, 95% CI = ±7.5%), while only half agreed that they would be comfortable learning in a group with EM trainees (50%, 95% CI = ±13%).

DISCUSSION

Although EM faculty perform and teach procedures, little is known about how they view skill attrition or their preferences for maintaining procedural competency. This may be due to the lack of available tools to survey skill competence or a reluctance of to engage in skills assessments that risk exposing procedural incompetence. ¹⁸ Our finding that rarely performed procedures are associated with lower faculty confidence suggests that the FPS survey demonstrated appropriate construct validity. Our results also show that faculty believe procedural skills decay is problematic in academic EM. Faculty acknowledged both observing and experiencing procedural skills attrition—especially for uncommonly performed procedures—suggesting that they are cognizant of the need for frequent, ongoing practice. This may reflect the fact that self‐assessment is, as Eva and Regehr ¹⁹ note, “a situationally bounded cognitive process that is context specific and dependent upon expertise.”

Finally, although the ideal method to prevent or reverse skills decay remains unknown, faculty indicated a preference for task trainers and simulation‐based learning. Simulation has been shown to allow practice with procedures in a safe environment and improved performance in the clinical environment. ²⁰ , ²¹ , ²² For educators designing faculty skills programming, these results provide some guidance to focus on simulation‐based learning in faculty peer groups.

LIMITATIONS

This study was conducted at a single site and may not represent the practice or attitudes of all academic EPs. For example, most EM faculty in our department work at multiple clinical sites, each with differing levels of trainee staffing. Thus, our structure may have contributed to the variation found among faculty in terms of estimated amount of trainee supervision. We are planning a multisite study using the FPS survey to examine if our conclusions are generalizable to other academic departments and identify solutions to address skill attrition.

CONCLUSIONS

The faculty procedural skills survey demonstrated construct validity and revealed that procedural skills attrition is prevalent within an academic emergency medicine faculty group. In developing a curriculum for procedural skills maintenance, faculty overwhelmingly preferred learning with their peers using simulation‐based methods. Future research will focus on a multisite study among emergency medicine faculty using the faculty procedural skills survey, determining the optimal methods for skills retention, and developing recommendations for psychological safety in faculty skills programming.

CONFLICT OF INTEREST

The authors have no potential conflicts to disclose.

AUTHOR CONTRIBUTIONS

Study concept and survey design: Brian Clyne, Hannah Barber Doucet, Linda Brown, Christopher Merritt, Rory Merritt, Elizabeth Jacobs, Elizabeth Jacobs, Gianna Petrone, Andrew Musits, Michael J. Mello. Acquisition of the data: Brian Clyne, Hannah Barber Doucet, Janette Baird. Analysis and interpretation of the data: Brian Clyne, Hannah Barber Doucet, Janette Baird, Andrew Musits, Linda Brown, Christopher Merritt, Michael J. Mello: Drafting of the manuscript: Brian Clyne, Rory Merritt, Andrew Musits, Michael J. Mello. Critical revision of the manuscript for important intellectual content: all authors. Statistical expertise: Janette Baird.

Supporting information

Data Supplement S1. Faculty procedural skills survey.

 

Clyne B, Barber Doucet H, Brown L, et al. Maintaining procedural skills for academic emergency medicine faculty: A needs assessment. AEM Educ Train. 2021;5:e10648. 10.1002/aet2.10648