Preparing Neurology Residents and Advanced Practice Providers for the COVID-19 ICU—A Neurocritical Care Led Intervention

Abstract

Background and Purpose:

With the surge of critically ill COVID-19 patients, neurology and neurosurgery residents and advanced practice providers (APPs) were deployed to intensive care units (ICU). These providers lacked relevant critical care training. We investigated whether a focused video-based learning curriculum could effectively teach high priority intensive care topics in this unprecedented setting to these neurology providers.

Methods:

Neurocritical care clinicians led a multidisciplinary team in developing a 2.5-hour lecture series covering the critical care management of COVID-19 patients. We examined whether provider confidence, stress, and knowledge base improved after viewing the lectures.

Results:

A total of 88 residents and APPs participated across 2 academic institutions. 64 participants (73%) had not spent time as an ICU provider. After viewing the lecture series, the proportion of providers who felt moderately, quite, or extremely confident increased from 11% to 72% (60% difference, 95% CI 49-72%) and the proportion of providers who felt nervous/stressed, very nervous/stressed, or extremely nervous/stressed decreased from 78% to 48% (38% difference, 95% CI 26-49%). Scores on knowledge base questions increased an average of 2.5 out of 12 points (SD 2.1; p < 0.001).

Conclusion:

A targeted, asynchronous curriculum on critical care COVID-19 management led to significantly increased confidence, decreased stress, and improved knowledge among resident trainees and APPs. This curriculum could serve as an effective didactic resource for neurology providers preparing for the COVID-19 ICU.

Introduction

The Coronavirus Disease 2019 (COVID-19) pandemic has overwhelmed hospitals and healthcare systems worldwide. Specialist providers in the neurosciences and specialized units such as neurological intensive care units (NICUs) rapidly transitioned to care for patients with COVID-19. Despite lacking critical care training, neuroscience (i.e. neurology and neurosurgery) residents and advanced practice providers (APPs) were tasked with managing ventilators, anesthetic drips, and acute respiratory distress syndrome (ARDS). Their lack of specific intensive care training became acutely relevant. Self-efficacy (i.e. the belief in our own ability to meet a given challenge) was low among these neuroscience providers while stress and burnout were high.¹ This created an urgent need to provide efficient and accessible critical care education to a large network of neuroscience providers, thus optimizing patient care and alleviating the stress of working in a new clinical setting.

Prior to the pandemic, increasing clinical and administrative demands led medical educators and learners to adopt a flipped classroom model.^2-4 This shift to pre-recorded, self-paced instruction has been found to improve learning efficiency and be as effective as traditional in-person education.^5-12 The COVID-19 pandemic further magnified the need for asynchronous video-based learning (VBL), accounting for high clinical demands, social distancing, and sheltering in place.

Our goal was to design a VBL curriculum to teach high-yield critical care concepts relevant to the management of COVID-19 to neuroscience residents and APPs without training in intensive care. To test the efficacy of our curricular intervention, we developed a survey measuring provider confidence, stress, and knowledge base. We hypothesized that the curriculum would decrease stress, increase confidence, and increase knowledge among neuroscience participants.

Methods

Study Design and Participants

This was a prospective study to examine the change in confidence level, stress level, and knowledge base after viewing the lecture series among neuroscience trainees and APPs from Weill Cornell Medical College-New York Presbyterian and the University of California, San Francisco. Trainees were neurology, neurosurgery, and internal medicine residents and PA students. APPs were neurology, neurosurgery, and internal medicine physician assistants. Over 60 percent of providers invited to take the survey were neuroscience providers. However, we did not ask participants to clarify their medical specialty in our survey so were unable to track the exact proportion of neurology as compared to internal medicine providers.

Standard Protocol Approvals, Registrations, and Patient Consents

The study was deemed exempt by the institutional review boards of Weill Cornell Medical College and the University of California, San Francisco.

Intervention

Participants electively enrolled in the study and were given access to the survey and lecture series. They were instructed to view the lectures in any order (Figure 1) and were asked to complete the pre-survey, lecture series, and post-survey within 14 days of receiving the invitation to participate in the study. The lecture series and post-survey were only available after participants completed the pre-survey. To estimate the number of days used for completing the surveys and watching the lectures, we calculated the duration of time between the submission time of the pre-survey and post-survey. Each participant was asked to create a unique identifier to maintain strict anonymity and link identities between the 2 surveys. Two reminder emails were sent out at day 7 and day 10.

Figure 1.

COVID-19 bootcamp lecture series.

Lecture Creation

Lectures and surveys were developed and modified in an iterative process by a multidisciplinary team that included neurocritical care and neurohospitalist attendings, neurocritical care fellows, neurology residents, neurosurgery APPs, and neurology APPs. Lectures were reviewed to ensure they had the appropriate content and level of difficulty. We first identified the critical care topics most relevant to the care of COVID-19: ventilator mechanics (2 lectures), ventilator weaning and sedation management (1 lecture), ARDS (2 lectures), shock (1 lecture), renal failure (1 lecture), cardiac arrhythmias (1 lecture), and pharmacologic management of COVID-19 (1 lecture) (Videos A-F). Attendings and fellows developed lecture content based on clinical experience, literature review, textbooks, and current guidelines from academic institutions and medical societies. An anesthesia critical care attending, who did not participate in the initial lecture development and was from a different institution, reviewed and revised the completed lectures to provide an objective assessment of content and difficulty. The final curriculum consisted of 9 pre-recorded lectures totaling 2 and a half hours which can be accessed via Figure 1. Additionally, a high yield summary document was created to review lecture content (supplemental Document 3).

Survey Creation

We created a survey to assess neuroscience participants’ experience, confidence, stress, and knowledge in caring for critically ill COVID-19 patients. The survey was given before (pre-survey) and after (post-survey) viewing the lectures (supplemental Document 1). The first section gathered information on the participant’s prior medical training and critical care experience and was only included in the pre-survey. The second section consisted of 12 questions assessing the participant’s level of confidence managing a critically ill COVID-19 patient across different clinical domains (e.g. ventilator management). Each question was scored on a five-point scale: (1) not confident, (2) slightly confident, (3) moderately confident, (4) quite confident, and (5) extremely confident. The third section consisted of 12 questions assessing the participant’s level of stress managing a critically ill COVID-19 patient across the same clinical domains. Stress was also scored on a five-point scale: (1) not nervous/stressed, (2) somewhat nervous/stressed, (3) nervous/stressed, (4) very nervous/stressed, and (5) extremely nervous/stressed. The fourth section consisted of 12 clinical vignettes with multiple choice questions focused on key learning points, one question per lecture and 3 additional questions covering airway management (supplemental Document 2). After the pre-survey and post-survey, participants were not provided answers to the 12 knowledge base questions, which prevented participants from memorizing answers and sharing answers with colleagues. In the post-survey, a fifth section consisted of 3 questions that assessed the perceived value of the lecture series. Neurocritical care attendings (7) and a fellow at Weill Cornell Medical College served as critical care experts and took the knowledge-based multiple-choice questions. Their average score was 11.5 out of 12.

Pilot Testing of Online Lecture Series

To obtain learner feedback, a pilot study was conducted in Spring 2020 among 23 volunteer NICU and neuroscience residents and APPs at New York Presbyterian-Weill Cornell. Pilot participants provided feedback on lectures and survey questions. We used their comments to adjust the survey questions and improve the clarity of lecture content.

Statistical Analysis

To describe the population, we examined participants’ confidence and stress levels at baseline (pre-survey) and after viewing the lecture series (post-survey). Within each clinical domain (ventilator basics, ARDS, cardiac arrhythmias, acute kidney injury, acute hypotension, cytokine storm syndrome, and COVID-19-associated coagulopathy), we calculated the median level of confidence and stress across all participants as an indication of group-level confidence and stress. For each participant, we then measured individualized levels of confidence and stress by calculating the median level of confidence and stress across all clinical domains (e.g. median level for ventilator management, cardiac arrhythmias, etc.) as an indication of a participant’s overall level of confidence or stress. Afterwards, we examined the change in a participant’s overall level of confidence and stress by calculating the difference between the overall confidence and overall stress reported by a participant in the pre-survey as compared to the post-survey. We also dichotomized confidence and stress to ease interpretability. We defined confident as a composite of “moderately confident,” “quite confident,” and “extremely confident” and not confident as a composite of “slightly confident” and “not confident.” We defined stressed as a composite of “nervous/stressed,” “very nervous/stressed,” and “extremely nervous/stressed” and not stressed as “not nervous/stressed” and “somewhat nervous/stressed.” Absolute proportion changes were calculated using McNemar test.

To evaluate the knowledge base of each participant, we calculated the proportion of clinical questions that the participant answered correctly in the pre-survey and in the post-survey. We used Student’s T-test to examine the change in confidence level, stress level, and knowledge base test score in the pre-survey as compared to the post-survey. Analyses were performed using STATA (Version 15.1, StataCorp, College Station, TX).

Data Availability

Any data not published within the article is available in anonymized form and will be shared by request from any qualified investigator.

Results

The pre-survey was completed by 170 participants and the post-survey was completed by 99 participants. We were able to link 88 pre-survey and post-survey submissions based on the identifier while the remaining 11 had unmatchable identifiers.

Of the 88 participants that had linked pre- and post-surveys, there were 54 (61%) PAs, 27 (31%) neurology, neurosurgery, and internal medicine residents, and 7 (8%) PA students (Table 1). There were 64 (73%) participants who had never spent time as the primary provider in an ICU and 15 (17%) participants who had spent one to 3 months (Table 1). Only 9 (10%) participants had spent time in a COVID-19 ICU, with 8 of them being PAs (Table 1). We found that on average, there were 4.2 days (SD, 3.8) between submission of the pre-survey and post-survey. The 82 participants who completed a pre-survey but did not complete the post-survey (or have a linkable identifier) had spent more time in the ICU, had a higher overall confidence at baseline (median 1.5; interquartile range [IQR] 1-2; p = 0.75) and a lower score on the knowledge base questions (p = 0.11) when compared to those participants that completed both surveys. However, none of these differences reached statistical significance.

Table 1.

Prior Intensive Care Unit Experience by Provider Type.

Prior ICU experience	All participants (N = 88)	APP (N = 54)	Resident (N = 27)	PA student (N = 7)
No time	64 (73%)	40 (74%)	17 (63%)	7 (100%)
1-3 months	15 (17%)	6 (11%)	9 (33%)	0
3-6 months	1 (1%)	0	1 (4%)	0
6-12 months	1 (1%)	1 (2%)	0	0
12-24 months	2 (2%)	2 (4%)	0	0
>24 months	5 (6%)	8 (15%)	1 (3.7%)	0

Abbreviations: APP = advanced practice provider, PA = physician assistant.

Confidence

At baseline, 78 (89%) reported feeling not or slightly confident on the 5-point scale evaluating their general ability to provide care for critically ill patients with COVID-19. After viewing the lecture series, 25 (28%) reported feeling not or slightly confident and 63 (72%) of participants reported being moderate, quite, or extremely confident. This corresponded to a 60% absolute increase of providers who felt confident versus not (95% CI, 49-72%; p < 0.001) (Table 2). The median level of confidence across all participants and all clinical domains also increased from 1 (IQR 1-2) to 3 (IQR 2.5-3) between the pre- and the post-survey (p < 0.001) (Table 3).

Table 2.

Overall Change in Composite Confidence and Stress Across All Providers.

	Pre-test (%)	Post-test (%)	Absolute proportion difference (95% CI)
Confident	11	72	60 (49 to 72)
Stressed	78	40	-38 (-49 to -26)

Confident = composite of “moderately confident,” “quite confident,” and “extremely confident”. Stressed = composite of “nervous/stressed,” “very nervous/stressed,” and “extremely nervous/stressed”.

Table 3.

Self-reported Confidence and Stress Levels Before and After Lecture Series For All Clinical Domains by Provider Type.

Participant type	Pre-test confidencea median (IQR)	Post-test confidencea median (IQR)	P-value	Pre-test stressb median (IQR)	Post-test stressb median (IQR)	P-value
All	1 (1-2)	3 (2.5-3)	<0.001	3.5 (2.5-4.25)	2 (2-3)	<0.001
APP	2 (1-2)	3 (3-3.5)	<0.001	3 (2-4)	2 (2-2.5)	<0.001
Resident	1 (1 -1)	3 (2-3)	<0.001	3.5 (2-4)	2 (2-3)	<0.001
PA Student	1 (1 -1)	3 (2-3)	<0.001	5 (4-5)	2.5 (2-3)	<0.001

^a Confidence rated on a 5-point scale (1 = not confident, 2 = slightly confident, 3 = moderately confident, 4 = quite confident, and 5 = extremely confident)

^b Stress rated on a 5-point scale (1 = not nervous/stressed, 2 = somewhat nervous/stressed, 3 = nervous/stressed, 4 = very nervous/stressed, and 5 = extremely nervous/stressed).

Stress

At baseline, there were 68 (77%) participants who reported feeling stressed, very nervous/stressed, or extremely nervous/stressed while the remaining 20 (23%) participants were not stressed (1%) or somewhat nervous/stressed (Table 3). After viewing the lecture series, stress level decreased with only 35 (40%) providers reporting that they were stressed, which corresponded to an absolute proportion decrease of stress by 38% (95 CI, 26-49%; p < 0.001) (Table 2). The median level of stress across all participants and clinical domains decreased from 3.5 (IQR 2.5-4.25) to 2 (IQR 2-3) (p < 0.001). On average, participant level of stress decreased 1.1 points (SD 1.0; p < 0.001).

Knowledge Base

At baseline, across all participants, 7 (59%) out of 12 multiple choice questions were answered correctly (Table 4). After viewing the lecture series, there was an average increase in correct answers by 2.5 questions (SD 2.1; p < 0.001) and overall correct answers increased to 9.6 (80%) out of 12 (Table 4). The highest increase in scores was noted among residents and PA students, 3.1 (SD 2.1; p < 0.001) and 3.7 (SD 1.0; p < 0.001), respectively (Table 4). Although both groups had a significant increase in knowledge, participants with no prior ICU experience had a higher increase in scores 2.8 (SD 2.2; p < 0.001) than those with >1 month experience 1.8 (SD 1.9; p < 0.001). The increase in knowledge was significantly higher in those with no prior ICU experience than those with > 1 month experience (p = 0.03).

Table 4.

Critical Care Knowledge Before and After Lecture Series by Provider Type and ICU Experience.

Participant type	Pre-test score mean (%)	Post-test score mean (%)	P-value
All	7 (59%)	10 (80%)	<0.001
APP	7 (61%)	9 (78%)	<0.001
Resident	7 (61%)	10 (86%)	<0.001
PA Student	5 (39%)	8 (70%)	0.001
ICU experience
No time in ICU	6.5 (54%)	9.6 (80%)	<0.001
> 1 month	8 (67%)	10.6 (88%)	<0.001

Abbreviations: APP = advanced practice provider, PA = physician assistant.

Series Evaluation

In evaluating the value of this lecture series, 85 (97%) participants reported that the lecture series improved their knowledge of managing patients with COVID-19; 78 (89%) reported that this lecture series improved their ability to care for critically ill COVID-19 patients, and 86 (98%) participants stated they would recommend this series to a colleague.

Discussion

This novel curriculum presented high-yield material in an asynchronous video lecture series to prepare non-ICU-trained neurology and neurosurgery residents and APPs to care for critically ill COVID-19 patients. Participating in the lecture series led to increased confidence, decreased stress, and improved knowledge base among non-ICU-trained-providers with neuroscience and internal medicine backgrounds. To our knowledge, this has not been shown in other asynchronous resources developed since the beginning of the COVID-19 pandemic. These findings have broad implications for preparing our healthcare workforce to care for patients with severe COVID-19 as well as other critical illnesses.

During this pandemic, medical providers have been under duress and there has been significant burnout. Increased self-efficacy has been linked to an ability to cope with stress and a decreased likelihood of developing burnout.^13,14 In our study, self-efficacy is extrapolated from our assessment of neuroscience providers’ confidence in caring for critically ill patients with COVID-19. Our curriculum significantly increased overall general confidence among these providers as well as confidence related to specific critical care topics such as ARDS and ventilator management. Given the association of improved provider self-efficacy and decreased stress, the study’s results warrant the use of this series.

The efficacy of this intervention was driven by the diversity in training and clinical experience among the research team. We had insight into the specific gaps in knowledge among non-ICU-trained providers. The iterative process was predicated on the specific needs of the neurology trainee and APP. The intended learners (i.e. neuroscience trainees and APPs) took part in developing the curricular content. The learning objectives, survey questions, summary sheet, and images were also informed by practical experience that the team had caring for patients with severe COVID-19 earlier in the pandemic. We also used formal feedback from a pilot study of neuroscience providers to further refine content and to ensure it met the specific needs of target learners.

Furthermore, the asynchronous learning structure bolstered the efficacy of the intervention. Prior research has shown that residents prefer teaching that is evidence-based, short in duration, structured around clinical cases or questions, and is directly applicable to patient care.^15,16 In this pandemic setting, we built our lectures around these concepts, choosing to create a short series given the time constraints of the target audience and to reduce the cognitive load. Although social-distancing and shelter-in-place protocols were being enforced, the asynchronous VBL platform ensured that neuroscience providers could rapidly access critical care training adapted to COVID-19. This curriculum was delivered using self-directed video based learning and could be used as a component in a flipped classroom model, both of which are ideal in a pandemic environment. These practical advantages likely contributed to nearly every participant (98%) noting that they would recommend this series to a colleague.

We note several limitations to this curriculum and this study. First, this resource was intended to be introductory and high yield, not comprehensive. There were many important topics on the management of COVID-19 that were not covered and its educational benefit may be higher for non-ICU as opposed to ICU-trained providers. Additionally, our study does not compare our educational intervention to another educational resource or format, thus we are unable to conclude that our series is more effective than other resources. Next, the lecture content assumes a baseline level of medical knowledge. Those without some inpatient medical knowledge (such as how to interpret an arterial blood gas) may find this series to be difficult. Given the urgency of the pandemic, the survey tool was not rigorously tested on an external group to evaluate the interpretation and validity of each question at measuring confidence and stress level. Furthermore, we implemented the curriculum in certain types of trainees at large academic institutions which may limit the generalizability to other non-ICU trained providers who practice in different clinical settings and are at a different stage in their career. We were unable to collect detailed information explaining why 82 participant were lost to follow up. Although not statistically significant, the providers who did not complete the post-survey had spent more time in an ICU and had a higher overall baseline confidence. They may have had decreased incentive to complete the lecture series or complete the post-survey questions. In addition, the volume of urgent clinical duties from the pandemic may have reduced available time to complete the post-survey. Finally, we do not know if increased confidence and decreased stress are persistent over time or cause an improvement in patient care.

Many recent publications report the infrastructural changes necessary for adapting neurology departments for the COVID-19 pandemic but do not address the educational needs of personnel. To successfully transition between a neurology service and one that is caring for critically ill COVID-19 patients is not only a matter of those important systems-based changes. Our curriculum successfully addresses the need for an adapted critical care resource that prepares non-ICU trained neurology residents and APPs for a COVID-19 ICU. This study specifically targets the practical medical experiences of neuroscience providers with the aim of improving their knowledge, preparedness, and safety. Although designed for our colleagues in the neurology and neurosurgery departments, our study similarly benefitted internal medicine residents and APPs. We hope that this lecture series, integrated in a flipped classroom setting or as a standalone didactic resource, will facilitate the preparation of neuroscience trainees and providers for service in a COVID-19 ICU. In this way, we might all meet the challenges of caring for critically ill COVID-19 patients with more confidence and capability.