Anesthesia Simulation Boot Camp—a Decade of Experience Enhancing Self-efficacy in First-year Residents

Abstract

Background:

Novice anesthesiology residents must acquire new technical, cognitive, and behavioral skills as they transition into the high-stakes perioperative environment. Simulation-based education improves procedural skill and behavior, and it permits deliberate practice with feedback; exposure to uncommon, high-consequence events; assessment; reproducibility; and zero risk to patients. We introduced a 5-day, high-fidelity Simulation Boot Camp (SBC) in 2006 for first-year clinical anesthesia residents (CA-1s) and report over a decade of experience assessing its impact on self-efficacy, value, feasibility, and sustainability.

Methods:

All CA-1s in our residency program participated in the SBC as part of orientation. Participants completed 2 individual high-fidelity simulations per day, each with a private debriefing session from an attending anesthesiologist in our simulation center. We measured their self-reported confidence, which we report as self-efficacy (SE), the belief in one’s own ability to successfully execute a skill or behavior necessary for a desired outcome, for 25 basic anesthesia skills before and after course completion. Participants also completed a postcourse evaluation.

Results:

Of the 281 CA-1s who participated in the course from 2006 to 2016, we collected data on 267 (95%). SE improved over the course of SBC for all 25 individual skills (P < .001) and remained stable over the decade-long period of study. Univariate analysis revealed a strong association between increased SE and male sex (P < .001), video gaming experience (P < .001), and completion of a prior residency (P = .018). Males were also more likely to report video gaming experience (P < .001). Multivariable analysis revealed that although women had lower SE than did men, they had a greater increase in SE attributed to participation in SBC (P = .041). Participants strongly agreed SBC was a realistic and nonjudgmental learning tool, built confidence, and should be mandatory. Most comments were positive, reflecting overall satisfaction with SBC

Conclusions:

SBC increases SE, is feasible, valuable to participants, and sustainable with remarkably consistency over the study period.

Introduction

Novice anesthesiology residents must quickly acquire new technical, cognitive, and behavioral skills as they transition into the high-stakes perioperative environment. Opportunities to practice are limited by patient-safety concerns, production pressure, duty hour restrictions, and Accreditation Council for Graduate Medical Education required clinic-based rotations and nonclinical requirements. Initial acquisition of procedural skills should be directed away from patients, shifting early training from traditional in-operating-room experiential learning toward simulation education that produces a more confident pretrained novice.^1,2 Simulation education also augments and standardizes exposure to emergency and rare-but-critical events that occur inconsistently.

Simulation is highly beneficial to knowledge attainment³ and trainee satisfaction,⁴ and it permits deliberate practice of procedural skills⁵ with no risk to patients.⁶ Anesthesiology-specific simulation training improves knowledge, skill, task completion time, and behavioral processes,^7,8 it and exposes learners to rare critical events^9–11 that require rapid diagnosis and treatment to mitigate patient harm.¹² Simulation is especially valuable in training for rare emergencies,^13–15 complex intraoperative tasks,¹⁶ skill completion, and management of intraoperative events.^11,17–19 Residents believe that simulation-based critical incident training is essential for safe anesthesia practice and agree that it should be required.²⁰

Self-efficacy (SE) theory addresses the difference between basic competency and the ability to apply skills while managing time-sensitive events under stress. SE is defined as the belief one holds in one’s ability to successfully execute a skill or behavior necessary for a desired outcome.²¹ SE is optimally measured through self-report and reflects the judgment an individual makes regarding the level of skill acquired and the strength of that belief. High SE is associated with greater perseverance, effort, and resiliency^22,23 and less self-doubt and anxiety when encountering challenges.²⁴ Conversely, low SE may hamper one’s conviction to perform familiar skills under stress, but can be enhanced.²² While physicians may have limited ability to self-assess competence accurately,²⁵ learners self-assess competency more reliably early in their training.²⁶

Boot camp courses enhance preparation for learners entering new roles.²⁷ Simulation boot camps improve knowledge, clinical skills, and confidence of medical and surgical residents during stressful training transitions,²⁸ and increases SE in anesthesiology trainees^29–31 and anesthesiology fellows.³¹ In 2006, we introduced a 5-day, high-fidelity Simulation Boot Camp (SBC) for first-year clinical anesthesia residents (CA-1s). The goal of our SBC was to enhance participants’ knowledge, skills, and competence in order to manage an intraoperative crisis until additional help arrives. We accomplished this with deliberate practice focused on basic anesthesia skills and communication during simulated crises. We report more than a decade of experience assessing the impact of SBC on SE, and on the value and sustainability of SBC.

Materials and Methods

Description of SBC Program and Simulation Environment

This study was approved by the Johns Hopkins Institutional Review Board, and informed consent was waived. Our institution is an academic tertiary care hospital in the northeastern United States with an anesthesiology residency program averaging 25 new CA-1s per year. CA-1s begin residency with a 4-week orientation period in the operating room (OR) under the constant observation of an attending anesthesiologist preceptor. All residents begin orientation July 1 in a single group. SBC is conducted during the third week of orientation, after basic familiarity with the perioperative environment has been achieved, and continues to be part of our orientation curriculum in 2020.

SBC is conducted in our simulation center, where simulated ORs are outfitted with surgical and anesthesia equipment mirroring our patient ORs. Laerdal SimMan adult manikins (SimMan Classic, 2006–2012; SimMan 3G, 2013-present; Wappingers Falls, NY) with full physiologic functioning are used. They allow for mask ventilation, intubation, auscultation of breath sounds, palpation of pulses, advanced cardiovascular life support, appropriate moulage or the application of mock injuries to enhance realism, and monitoring of echocardiogram, pulse oximetry, invasive and noninvasive blood pressure, end-tidal CO₂, and temperature.

Our SBC education team consisted of 10 board-certified practicing anesthesiologists with a minimum of 3 years’ interest and experience in simulation education, although no formal training or certification was required. The faculty assigned each day were relatively consistent throughout the 10-year study period, although some substitutions did occur. Our team also included 4 OR nurses and/or scrub technicians, 2 certified simulation education specialists, and a simulation technical specialist. A subset of the most experienced anesthesiologists acted as simulation operators, manipulating manikin responses from a concealed adjacent control booth. The operator used earpieces to communicate with confederates in the OR, including a circulating nurse, scrub technician, and surgeon(s) (performed by an anesthesiologist), each of whom interacted with the participant as part of an OR team. Team members provided additional details about the patient; carried out typical roles and responsibilities; created an authentic soundscape of ambient noise; and assisted, encouraged, or challenged the participant’s decision-making.

Before SBC, participants received a 1-hour simulation overview lecture that reviewed the manikin’s capabilities. Expectations for the course were reviewed, including the participant’s role as a CA-1 under the supervision of an anesthesia attending. Participants were informed that they would be videotaped but they could choose to decline. Videotaping is frequently used at our simulation center but is not routine. In this context, video was used primarily to confirm the participant’s actions that may not have been verbalized so the operator could respond appropriately, and the recording of any individual’s performance was available to participants upon request. Participants were also informed verbally that SBC was purely an educational experience, not an evaluation, and their performance was confidential and would not be shared with peers, other faculty, or their program director. The SBC faculty agreed that performance in the simulation environment might not accurately reflect performance in a clinical environment because risk-taking behavior in the absence of risk of harm to patients may be different. As such, the faculty had a tacit agreement that events in the simulation center were to remain exclusively in that context which was not regulated in any formal way.

Each CA-1 was relieved from the OR for 1 hour each day during SBC week to participate in 2 individual intraoperative simulations and personalized debriefing sessions as a single participant, for a total of 5 hours over 1 week, including prebriefing and travel time. Each simulation lasted 20 minutes total (10 minutes of simulation time, 10 minutes for debriefing), and all participants completed the same 2 scenarios on any given day. This schedule required more than 1 faculty member in the role of anesthesiologist so that debriefing with the first participant could occur while the subsequent participant was introduced to the scenario. The sessions were scheduled to rotate at different times each day to minimize disruptions to clinical activities (Supplemental Online Material, Appendix A).

The attending anesthesiologist facilitated the scenario, introduced the case, and gave the participant scripted materials containing information on patient history, physical exam, and ongoing anesthetic record if the case was in progress. The attending supervised airway management if appropriate, provided concurrent feedback on technical skills, and then departed the OR to view the participant from the control booth. Participants were asked to verbalize any medications and doses, fluids, or blood products administered, and these actions were verified with video. Simulations were scripted, but the manikin’s response was controlled by the operator in real time based on the participant’s management of the scenario. Physiologic urgency was modified depending on the level of competence and stress demonstrated by the participant. At the conclusion of the scenario, the attending returned to the simulated OR to receive the participant’s interpretation of events, offer assistance or constructive criticism, and guide the participant through the process of stabilizing the patient if necessary. Immediately afterward, the attending facilitated a private debriefing session with the participant. The attending solicited the participant’s assessment of their performance and reaction to events, provided feedback on clinical management, and informally assessed the level of emotional stress by inquiring how SBC and orientation were going. Learning objectives of the scenario were reviewed and included aspects of Anesthesia Crisis Resource Management skills such as leadership, obtaining help from available resources, team dynamics, and recognition and prevention of fixation errors.³² The debriefing was not scripted but began with open-ended questions designed to elicit the participant’s reflection on their own performance and determine if they understood the clinical issue presented. This was generally followed by a conversation where the participant asked questions about that scenario and its relation to their varied clinical experiences. Debriefing did not use the videotape of the performance.

A deliberate practice methodology was used to reinforce skills and behavior applicable to the simulated cases in an attempt to generate a more instinctual response to critical events. Each day, one scenario focused on airway management beginning with induction of anesthesia and intubation, followed by desaturation of oxygen as a result of several etiologies (Table 1). This tactic provided an iterative experience on a single theme with escalating complexity throughout the week. The other scenario each day focused on common physiologic issues that occurred during a case already in progress with a similar escalation of intensity. On the final day, the residents worked in pairs to manage an intraoperative cardiac arrest and adapt advanced cardiovascular life support protocols and team roles to the OR. The design of each scenario embedded a forced error model, and content was informed by the ACGME core competencies and the list of essential anesthesia skills queried in the SE survey (Supplemental Online Material, Appendix B).

Table 1.

High-fidelity Simulation Schedule

Days	OR 1: Airway Simulations	OR 2: Physiology Simulations
Monday	1: Mainstem endobronchial intubation—addresses oxygen desaturation and unilateral breath sounds in healthy patient. Requires simple correction of withdrawing ETT to proper depth.	2: Wrong-site surgery—addresses the patient safety issue of wrong-sided surgery in an anesthetized patient. Requires a discussion on how to proceed once the error is realized.
Tuesday	3: Cuff laceration with circuit leak—addresses oxygen desaturation and abnormally high compliance with manual ventilation. Requires ETT exchange in a patient at high risk for aspiration and rapid sequence induction at the beginning of the scenario.	4: Hypovolemia due to hemorrhage—addresses signs of shock, hypotension, and tachycardia, which would be recognizable from previous medical training, resulting from significant blood loss during surgery. Requires managing resuscitation, administering blood products, vasopressors to anesthetized patient.
Wednesday	5: Bronchospasm—addresses oxygen desaturation and abnormally low compliance with manual ventilation due to an obstructive process. Requires generation differential diagnosis in complex patient with many possible causes, selecting an appropriate treatment modality including mechanics of administering inhaled albuterol to an intubated patient.	6: Bradycardia from peritoneal retraction—addresses a vagal response indicated by sudden bradycardia and hypotension due to surgical manipulation, conducted with a high degree of time pressure in a situation that may be less familiar outside of the OR or from previous medical training at the postgraduate year-2 level.
Thursday	7: Inadvertent extubation—addresses oxygen desaturation and loss of end-tidal CO2 in patient with head 180° away from anesthesia provider and covered by sterile drapes for ongoing thyroid surgery. Very complex crisis situation requires management of complete airway loss caused by inadvertent removal of ETT while patient is in a challenging positioning, and convincing surgeon to disturb the sterile field so patient can be reintubated.	8: Light anesthesia in patient with cardiac disease—addresses tachycardia and hypertension resulting from surgical stimulation in an inadequately anesthetized patient with cardiac disease. Requires a discussion with the surgical team to pause surgery while correcting the problem by deepening the anesthetic, supporting blood pressure, controlling heart rate, and addressing the cardiac ischemia seen on the monitor as a result of the physiologic perturbation.
Friday	9 (9a): ACLS - cardiac arrest, local anesthetic toxicity (1st responder); (9b) (2nd responder); participants work in pairs for this scenario - addresses an intraoperative cardiac arrest from local anesthetic toxicity in a patient with complex past medical history and an intraoperative course that provides the basis for a broad differential diagnosis for the cause of the arrest. This is the most complex and time-pressured scenario and is the culmination of multiple skills addressed during Simulation Boot Camp week: hemodynamic perturbation; cardiac arrhythmia; effective communication in a crisis; rapid assessment of airway, breathing (reinforcing skills learned in the airway room), and circulation; the opportunity to discuss roles in the OR during an arrest; high-quality cardiopulmonary resuscitation; use of the defibrillator; ACLS pharmacology; and generating a differential diagnosis for cardiac arrest.

Abbreviations: ACLS, advanced cardiovascular life support; ETT, endotracheal tube; OR, operating room.

Data Collection

The data collection period was July 2006 to July 2016. All participants had the option to submit data anonymously. All participants completed a precourse demographic survey that included information about prior postgraduate medical training and experience with simulation education. Video gaming experience was also queried based on evidence that video games can enhance task-specific SE.³³ The participants completed a precourse SE survey rating their confidence in their ability to perform 25 essential anesthesia skills on a scale of 1 (judgment of no confidence) to 10 (judgment of complete confidence) just prior to the first scenario. We did not ask participants to rate their strength in that belief. To create the SE survey, 4 simulation faculty members, including authors E.J. and B.L., used a modified Delphi approach^34,35 to generate a list of common skills crucial for novice anesthesiology residents to master before completing orientation. These items were subject to 2 additional rounds of selection and modification before a final list was approved by all members. SE skills were further categorized into 3 subcategories: technical skills, diagnosis/situational awareness (diagnosis), and treatment/management skills (management). Immediately after completion of SBC, participants filled out the same SE survey and completed a postcourse evaluation with optional anonymity (Supplemental Online Material, Appendix C). All materials were paper-based. Missing data are the result of participants who did not answer some questions, lost data sheets, or anonymity not permitting SE surveys and postcourse evaluations to be linked.

We conducted thematic analysis of free-text responses using the methods of Braun and Clarke.³⁶ One author (CM) grouped the comments into positive and negative categories, generating an initial set of thematic codes, and reviewed responses to verify that none were overlooked. Each response received 1 point for each thematic code it contained. More than 1 code was possible for responses containing multiple themes, and some participants made multiple comments. A second author (AS) verified the accuracy of the coding. Discrepancies were discussed until consensus was achieved.

Statistical Methods

This study is largely exploratory. There are no prior data to facilitate powering of the study. Summed self-efficacy (SSE) is the sum of all 25 10-point component SE scores. We identify SSE as the primary outcome variable, for which each subject serves as their own control.

In any given study year (typically 25 participants), assuming a standard deviation of the differences in SSE of 20% its maximal value of 250, at a significance level of 0.05 and power of 0.9, the study should be able to detect a difference in SSE of 33.8, an effect size of 0.68. Over the decade course of the entire study, where 250 participants might be expected, the study should be able to detect a difference of 10.3, an effect size of 0.21. Since the SSE is the sum of 25 10-point component SE scores, assuming the same 20% of the maximal score of 10, and significance level of 0.05 and power of 0.9, a difference of 0.41 points should be detectable, an effect size of 0.21. We performed additional secondary analyses to relate SSE to the demographics of the study population, particularly after identification of significant differences in uniformity in the study population.

Mean and standard deviations are reported for continuous variables, whereas proportions in percent are given for categorical data. The binary probability test was used to determine the extent that demographic parameters were equally distributed within the population. Frequency data were analyzed with a Fisher exact test. Data from postcourse evaluation surveys, scored on a 5-point Likert scale, were treated as ordinal data and confirmed with continuous analysis. Data from SE surveys on a 1 to 10 scale were treated as continuous variables. Data for each question from the SE survey were totaled to generate an aggregate, or SSE (maximal score of 250), for each subject before and after SBC participation, the study’s primary outcome variable. All available data were analyzed, not just linked presurvey-postsurvey pairs. These data were analyzed by mixed model regression with random intercepts. After determining the net impact of SBC, we examined the influence of demographic variables and the association with course evaluation survey results using the same approach. Regression with continuous variables was used to evaluate trends in SSE over the entire decade of the study. Similarly, logistic regression was used to evaluate trends in course evaluations over the entire decade of study with these results confirmed with continuous analysis. All significance values were obtained from 2-sided tests, and P < .05 was used as the threshold for significance. All significance values were interpreted while keeping multiple comparisons in mind. All surveys were deidentified. Data were entered into an Excel spreadsheet (Microsoft Corp, Redmond, WA), and manipulation was facilitated with Mathematica version 11 (Wolfram, Inc, Champaign, IL). Statistical analysis was facilitated with Stata version 13 (Statacorp, College Station, TX).

Results

Study Population and Demographics

All new CA-1s entering our program from 2006 to 2016 participated in SBC. Figure 1 shows the number of participants and completed datasets. Demographic data were available for 254 participants (Table 2), where the significance of differences from a perfectly uniform population is given for each parameter. The number of participants who lacked prior experience with simulation education decreased over the study period (odds ratio: 0.74, P < .001). Overall, more men than women reported video gaming experience (n = 62 vs 14, P < .001).The number of participants having had no prior residency experience increased (odds ratio: 1.137, P = .026) but was not associated with sex (P = .60). No other demographic data changed over the study period.

Figure 1.

Diagram of the total number of participants in Simulation Boot Camp, the number of participants who completed each of the data collection tools, and the complete set of comparison data analyzed in this study. Abbreviation: CA-1, first-year clinical anesthesia resident.

Table 2.

Demographics of First-year Clinical Anesthesia Resident (CA-1) Study Participants (N = 254)

Demographic	Frequency, n (%)	P Valuea
Participant characteristics
Sex		.57
Female	122 (48)
Male	132 (52)
Age, years		<.001
<25	0
25–30	177 (69.7)
>30	74 (29.1)
N/A	3 (1.1)
Previously completed postgraduate medical training		<.001
Yes	41 (16.1)
No	182 (71.6)
N/A	31 (12.2)
Prior exposure to training with a high-fidelity manikin		<.001
Yes	153 (61.2)
In medical school	133 (87)
In internship	66 (43)
No	97 (38.2)
N/A	4 (1.5)
Type of internship completed		<.001
Internal medicine	157 (61.8)
Transitional	42 (16.5)
Surgery	17 (6.8)
Pediatrics	17 (6.8)
Other or N/A	21 (8.2)
Play video games		.026
Yes	76 (29.9)
No	107 (42.1)
N/A	71 (27.9)
Play massively online role-playing game		<.001
Yes	11 (4.3)
No	168 (66.1)
N/A	75 (29.5)

Abbreviation: N/A, not answered or not applicable.

^a Indicates probability of observing the distribution if the population was actually equally distributed as determined by the binomial probability test. When more than 2 groups were present, the largest was compared against the sum of the others. N/A was not included in the determination of significant differences.

Self-efficacy

SE improved over the course of SBC for every one of the 25 individual skills (P < .001, individually Figure 2). Rapid sequence induction and treatment of bronchospasm had the lowest pre-SBC SE scores. The greatest impact of SBC on SE was in the treatment subcategory, increasing an average of 1.84 points per skill, with treatment of bronchospasm showing the largest increase (P < .001). Overall, SSE scores increased by 38.0 (95% confidence interval: 34.6–41.4, P < .001) after completion of SBC when compared to the pre-SBC value of 131.5 (95% confidence interval: 126.9–136.1) out of a possible maximum of 250.

Figure 2.

Bar graph showing the change in self-efficacy (SE) scores for each of the 25 essential anesthesia skills taught during Simulation Boot Camp (SBC). The SE for each skill was solicited on a discrete 1 to 10 scale. The mean (SD) for each is displayed for the data obtained from the survey before SBC start (light gray bars, maximum n = 252 [250–252]) and after its completion (dark gray bars, maximum n = 249 [245–249]). The individual skills are listed on the light gray bar, and the associated survey is provided in Supplemental Online Material, Appendix B. Skills are grouped into technical skills/tools, diagnosis/situational awareness, and treatment/management for ease of comparison. The horizontal line represents an SE score of 5 for reference. Overall SE increased for each skill (P < .001) over the course of the SBC. Improvement for each category is shown underneath the appropriate section and represents the average improvement per skill in each category. Abbreviations: ETT, endotracheal tube; IV, intravenous; HTN, hypertension; RSI, rapid sequence induction.

SE as a Function of Demographics

Univariate longitudinal analysis comparing SSE obtained at the beginning and end of SBC revealed a strong association between higher SSE and male sex (P < .001), video gaming experience (P < .001), and completion of a prior residency (P = .018) as indicated by the significant differences in their intercepts (Figure 3). Although the intercepts were significantly different, we observed trends toward increases in the amount of change (slope) in SSE (female: P = .06, no video games: P = .12, no prior residency: P = .08) attributed to SBC. Multivariable analysis (Figure 3D), motivated by these findings and the significant differences in men and women who played video games (P < .001; Table 2), revealed that these 3 variables continued to be significant. Although SSE scores were lower for women than for men, the trend toward a greater increase in SSE demonstrated with univariate analysis reached significance with multivariable analysis that considered possible interactions (P = .041).

Figure 3.

Aggregate or summed self-efficacy (SSE) scores before and after residents participated in the Simulation Boot Camp (SBC) are shown as a function of sex (A), video game experience (B), and completion of prior residency (C). The maximum possible score was 250. For each point, the mean and standard error obtained from mixed model regression with random intercepts is given along with the P value for the differences in the intercepts. Although the intercepts were significantly different, the amount of change (slope) in self-efficacy (SE) scores over the course of assessment did not reach statistical significance (female: P = .06; video gaming experience: P = .12; prior residency: P = .08). The addition of an interaction term between sex and video gaming experience did not improve the model. The corresponding multivariable model of change in SE score is given in the accompanying table (D) (see text for details). Mixed model regression over time with random intercepts showed statistical significance. Interaction terms were included only if significant.

When multivariable analysis was performed separately for the 3 skill subcategories, small but significant effects were observed. Video gaming experience had a positive effect on all 3 skill subcategories (P ≤ .025). Male sex had a positive effect on technical skill (P = .004), and prior residency had a positive effect on diagnosis and management subcategories (P ≤ .025).

Longitudinal analysis revealed no change over the decade-long study period in the pre-SBC SSE scores (P = .55), although we observed a trend toward an increase in post-SSE of approximately 1 point per year (P = .06). Similarly, during the study period there was no change in the interaction between demographic variables or their relative influence on SSE (P = .55).

Postcourse Evaluations

In the SBC postcourse evaluation, participants strongly agreed that SBC was a realistic, nonjudgmental learning tool that supported what they were learning in orientation lectures, built confidence, and should be mandatory (Figure 4). They also considered the debriefing sessions to be useful. Participants disagreed with statements that SBC was unrealistic, too simplistic, or too difficult; that the timing was inappropriate; or that there was too much exposure to simulation-based learning during orientation. Statements indicating that the respondent felt embarrassed, self-conscious, or pressured were generally neutral but demonstrated high response variability. Longitudinal analysis of evaluation data revealed no temporal patterns.

Figure 4.

Bar graph showing results from the postcourse evaluation taken after Simulation Boot Camp. The response to each question was given on a 5-point Likert scale (−2 = strongly disagree, −1 = disagree, 0 = neutral, +1 = agree, +2 = strongly agree). The percentages of each response are shown graphically and total to 100. The percentage of participants responding to each category is presented in the bar as dark (neutral = 0), medium (moderate = −1, 1), and light colored (strong = −2, 2). The evaluation questions are provided in Supplemental Online Material, Appendix C.

To determine if postcourse evaluation questions were associated with highly variable responses and SSE scores, we analyzed the subset of linked complete pre-SSE and post-SSE and postcourse evaluations. Those who responded that they felt increased confidence or felt self-conscious on the postcourse evaluation, had higher (P = .017) or lower (P < .001) SSE scores, respectively. We observed trends toward decreased post-SSE scores in individuals who endorsed feeling pressured (P = .12) or embarrassed (P = .06) on the postcourse evaluation. There was no association between demographic factors that influenced SSE scores and the statement it gave me confidence to take care of real patients, though most participants agreed with the statement.

Thematic Analysis of Free-text Responses

Of the 267 participants that completed the evaluation, 253 made comments in one or more of the 3 free-text questions with a total of 651 individual responses, some of which contained multiple positive and/or negative themes, totaling 698 theme-coded responses. Of these, 481 of the 698 (69%) theme-coded responses were positive, reflecting overall satisfaction with SBC:

“I liked the chance to experience real emergencies that are stressful in nature and have the chance to react without making mistakes on real patients.”

“The scenarios were realistic and there was immediate feedback; it made me feel more confident that I could handle these situations intraoperatively.”

“On each session I can definitely say I learned a lesson that could be applied in clinical practice.”

Overall, 217 of the 698 (31%) theme-coded comments were negative. One-third of the negative comments reflected increased stress or embarrassment, however of these, almost half were qualified by statements that these aspects were necessary for realism or a valuable motivator for learning:

“I did feel embarrassed about my performance initially, but it was worth it as I truly felt like I learned crucial and applicable knowledge.”

“Stressful during actual scenario, but probably necessary to make it more realistic.”

“Initially very nerve-racking, especially the first session, but after that it got better—the preceptors were very supportive and made the experience conducive to learning.”

Discussion

We found that participation in SBC increased SSE, our primary outcome variable, because of improvement in SE for each of 25 essential anesthesia skills whose sum constitute the SSE. Secondary analysis revealed that male sex, video gaming experience, and prior residency positively influenced SE at baseline and post-SBC. Postcourse evaluations support SE scores and demonstrate the value of SBC to the novice anesthesia resident. Free-text comments were generally positive and consistent with quantitative data.

New residents in anesthesiology encounter a variety of challenges that are distinct from those of internship. They must integrate and apply cognitive and technical skills and master new equipment and techniques in the complex perioperative environment. They must also develop the experience and confidence to respond instinctively to a variety of critical events, physiologic perturbations, and patient crises. Clinical opportunities for learning these skills occur randomly and infrequently, and hesitation to diagnose and treat may pose significant risk to patients. Supplementing the OR learning experience with simulation training provides the novice with opportunities to develop skills, behaviors, and confidence to effectively manage critical events safely, shifting the dangerous early part of the learning curve away from patients.^1,27 Simulated clinical crises also allow residents to learn what adverse outcomes result from delayed diagnosis and treatment, without harming patients.³⁷

Our results are consistent with prior studies that support the effectiveness of simulation training in novice anesthesiology and critical care trainees.^29–31 However, these prior studies are limited in scope to 1 or 2 years and indirectly measure SE on a subset of tasks. We measured SE strength using a unipolar, task-specific scale that is the preferred method of measuring SE³⁸ and more predictive of performance ability^39,40 than a bipolar Likert scale. The length of our study and the number of participants permitted us to relate demographic factors such as sex, prior residency, and video gaming to SE. The lack of temporal variation in SE scores and postcourse evaluations indicate the consistency of the simulation environment over the study period.

SE is the belief one holds that s/he has the capability needed to organize and perform a task successfully rather than a more global trait accounting for overall performance optimism or self-confidence.^21,38,41 Beyond what can be measured objectively as actual skill level, these beliefs are thought to have a greater impact on motivation, emotions, and behavior.⁴⁰ Students with higher SE are more likely to seek challenges than shy away from them, persist while facing those challenges, and adopt effective strategies to mediate them. However, SE has a positive impact on behavior only if the proper incentives and necessary skills are present.²² Lack of both experience and confidence while performing under stress jeopardizes the execution of these skills during a crisis. SBC provides the opportunity for novice residents to increase SE and practice integration of these skills in a pressured but safe environment. This pretraining encourages sufficient conviction in their ability to address crises in the OR until help arrives.

Whether increased SE leads to better performance of technical skills or outcomes as it pertains to the health professions⁴² is often questioned. Meta-analyses provide clear evidence of a positive relationship between SE beliefs and work performance.^43,44 Increased SE leads to learners with more motivation,⁴⁵ more resiliency,^22,23 and less self-doubt and anxiety when encountering new or unexpected situations.²⁴ Studies in sports psychology have shown SE to have a positive correlation with performance.^46,47 The practice of anesthesia requires heightened situational awareness, integration of cognitive skills, procedural dexterity, communication, and teamwork, all in a time-pressured environment. Similar to sports, skill building does not occur in isolation but requires drilling and practice in a team format for effective integration. While self-assessment is not well correlated with observed measures of competence,²⁵ when combined with skill competency assessment by faculty, SE may be a good indicator of readiness for increased autonomy in the OR.

We found that SE was influenced by sex, video gaming experience, and completion of a prior residency. Gender disparity in medicine is well-known and multifactorial,^48–51 and anesthesiology is no different in this regard.^52,53 Discrimination may lead to lower SE, which in turn hinders self-regulated learning, professional growth, and achievement, and can damage motivation and social and emotional well-being.⁵⁴ In our study, women had significantly lower baseline SE than did men, particularly in the technical skill subcategory. In self-reporting instruments, women tend to self-judge more modestly than men and use different internal metrics.⁵⁵ It remains unclear whether men overreport or women underreport SE. We also show that SBC enhances SE in women to a greater degree. We posit that programs like SBC are effective at increasing SE because the individualized nature mitigates perceived gender-based discrimination. Thus, this style of learning may be particularly beneficial to women. Modalities known to help women are those that also increase SE, especially in STEM-based academic areas.⁵⁶

Video gaming experience can enhance task-specific SE³³ and certain cognitive functions, such as problem-solving skills, and increase motivation for task completion to approach mastery.⁵⁷ Video gamers may be more likely to accept simulation as a valid educational tool, more adept at suspending disbelief and overlooking minor inconsistencies, and more likely to prefer this type of learning. We show that video gamers have higher SE scores than nongamers independent of the coexisting gender disparity among video game players. We propose that repeated drilling, acceptance of failure, and the opportunity to reset in a consequence-free environment of gaming builds resiliency that is integral to SE. Video gaming is inherently a deliberate practice because it requires repetitive action and enhanced motivation to keep playing despite increasing difficulty, which leads to increased SE as one approaches mastery at game tasks.⁵⁸ Video gaming not only develops dexterity^59–61 but additionally spatial skills such as attention allocation and visual processing that have lasting benefits that translate outside the video game context⁶² supported by evidence of neural changes in the frontoparietal network demonstrated on functional magnetic resonance imaging studies.⁶³ Enhanced ability to attend to visual cues⁶⁴ may be translatable to the simulated environment in the form of increased situational awareness and vigilance.

Anecdotally, many participants who completed a previous residency had experience as pediatric intensive care unit fellows in an environment that required integrated teams to perform under time pressure. Whereas video gaming experience increased confidence in all 3 anesthesia skill subcategories, prior residency increased confidence in the diagnosis and management subcategories specifically, areas that may have greater overlap with other medical subspecialties, particularly intensive care, and that are distinct from anesthesia-specific technical skills, such as managing a vaporizer.

Although high-fidelity simulators are not essential to adding value, at least in teaching motor skills,⁷ the use of appropriate moulage can enhance enthusiasm and interest of learners,⁶⁵ and ambient noise contributes to realism by increasing task load.⁶⁶ We intentionally incorporated ambient noise of OR equipment, conversation, and requests from surgeons in order to simulate a realistic environment and engage the learner in an authentic OR experience. These techniques are also used to draw the learner out of a hypervigilant state where they focus on changes in monitor vital signs to the exclusion of normal perioperative team interactions. Whether participants felt the soundscape reflected a realistic environment or not, many were aware of an increased cognitive burden. Future studies will focus on the quantitative assessment of cognitive load in our simulation program.

The debriefing sessions were highlighted as particularly useful when assessed using a Likert scale and from free-text comments. Debriefing and feedback on performance are considered the most important aspects of simulation-based medical education as intervention or assessment.^67,68 However, video-based debriefing has not been shown to be helpful and could be detrimental.⁶⁸ Some participants suggested that viewing their own performance would be helpful, but it is not part of our debriefing process.

We collected data on 267 participants (95%) and linked paired pre-SBC and post-SBC SSE scores for 239 of them. We linked these results with demographic data in two-thirds of participants because of the limit imposed by the option to submit data anonymously. Missing data are the result of paper-based surveys where questions were not answered or a few lost surveys. Only 183 (72%) participants who answered the demographics survey responded to questions on video gaming, which we believe influences our primary outcome. The influence of incomplete data may be minimized by the 11-year observation time and the consistency of the data over the study period.

There are several limitations to this study. Although our metric inquired about confidence in performing a skill, we did not ask participants to rate their strength in this belief, and Bandura^21,22 suggests that both are components of SE. Furthermore, although SE may be related to improved performance, we did not measure this outcome. We believe that repetitive practice of SBC leads to increased situational awareness and SE that empowers participants to activate various levels of support, depending on the specific situation, and communicate with confidence and clarity in a crisis.⁶⁹ Future studies are warranted to examine the relationship of task-specific SE to task performance and cognitive burden. Future studies using this program as a platform will involve comparison of specific simulation methods and effect on task performance as well as the relationship between SE and concrete measures of competency such as examination scores and faculty evaluations. This was not a randomized controlled trial since the entire population and not a selected subset served as their own controls to assess the efficacy of SBC on SSE. While sex was uniformly distributed in our study population, it was not possible to rely on randomization to achieve a balanced study population for other asymmetric parameters such as video gaming experience. While other studies specifically designed to examine the relationship between video gaming and SE demonstrate that superior skills are translatable to other environments, future studies are needed to determine if these associations also apply to performance in anesthesiology training. Although simulation may be effective, it is also very expensive and labor-intensive. SBC is conducted over 5 consecutive days, and our faculty are paid for their time. We recognize that this level of support may not be possible for many programs and may limit generalizability of our method. This study does not compare simulation to other teaching methods or a no-intervention group, nor does it consider cost. We approached this program from an education prospective primarily and, like others, feel it could be inappropriate to have a no-intervention group in this environment.⁷⁰ It is possible that our preintervention and postintervention survey method could result in a response shift bias when reporting SE. Although using a retrospective preintervention and postintervention could minimize this bias, potentially leading to a larger change in SE than we report, it could also lead to greater variability.^71,72 Our post-SBC measures were performed immediately after course conclusion and do not assess a persistent effect on SE beyond the completion of SBC, although future studies will examine this.

In conclusion, SBC is valuable to participants and is a sustainable program with consistent results obtained over more than a decade. SBC increased SE in novice anesthesiology residents for all 25 skills measured. Participants who were male, had video gaming experience, and/or completed a previous residency had higher baseline SSE. Female participants appeared to derive more SE benefit from SBC. Although maintaining SBC is feasible for our residency program, our experience may have limited generalizability for other programs. Future studies will address whether increased SE correlates with improved performance in the simulation environment and the relationship with cognitive load.

Supplemental Online Material

Appendix A. Sample Participant Schedule for a Day in Simulation Boot Camp

Timeframe	Prebrief	Scenario 1	Debrief 1	Scenario 2	Debrief 2
7:20–7:30	1 & 2
7:30–7:40		1		2
7:40–7:50			1		2
7:50–8:00		2		1
8:00–8:10	3 & 4		2		1
8:10–8:20		3		4
8:20–8:30			3		4
8:30–8:40		4		3
8:40–8:50	5 & 6		4		3
… Participants 7–22 …
2:40–2:50	23 & 24
2:50–3:00		23		24
3:00–3:10			23		24
3:10–3:20		24		23
3:20–3:30	25		24		23
3:30–3:40		25
3:40–3:50			25
3:50–4:00				25
4:00–4:10					25

Appendix B. Summary of Self-efficacy Survey Questions Scored on a 1–10 Scale

Rate Your Confidence in Performing the Following Skills:
1. Administration of intravenous induction agents
2. Preparation of equipment for intubation
3. Flow meter and vaporizer manipulation
4. Bag-mask ventilation
5. Direct laryngoscopy
6. Recognition of landmarks for endotracheal tube placement
7. Placement of endotracheal tube
8. Confirmation of correct endotracheal tube placement
9. Rapid sequence intubation
10. Application of cricoid pressure
11. Recognition of adrenergic response to intubation
12. Management of adrenergic response to intubation
13. Recognition of end-tidal CO2 tracing abnormalities
14. Emergency management of hypoxemia
15. Emergency management of hypotension
16. Emergency management of hypertension
17. Emergency management of tachycardia
18. Assessment of hypovolemia
19. Emergency management of hypovolemia
20. Use of vasopressors and/or inotropic drugs
21. Emergency management of bronchospasm
22. Recognition of endobronchial intubation
23. Use of routine monitors at induction of anesthesia
24. Interpretation of monitors during maintenance of anesthesia
25. Radial arterial line monitoring

Appendix C. Summary of Postsimulation Boot Camp Survey Questions

Scored Using 5-point Likert Scale: Strongly Disagree to Strongly Agree (−2 to 2)
26. The simulation sessions were a realistic portrayal of clinical events
27. I was pressured to make decisions without enough information
28. The sessions were too simplistic
29. I did not have adequate information to make decisions
30. The manikin’s responses to events were realistic
31. Simulation-based scenarios are a good tool for learning clinical anesthesia principles
32. Simulation-based scenarios reinforce clinical concepts I have learned in lectures
33. Simulation should be a mandatory part of the CA-1 curriculum
34. Simulation-based scenarios were a generally useful experience
35. Simulation-based scenarios were a generally good experience
36. The sessions gave me confidence to take care of actual patients
37. The sessions were not very realistic
38. There should be more exposure to simulation during the CA-1 orientation
39. There was too much exposure to simulation during the CA-1 orientation
40. The simulation sessions were too short
41. The simulation sessions were too long in duration
42. The simulation sessions were too difficult
43. The debriefing sessions were generally useful
44. The debriefing sessions focused on systems issues and were non-judgmental
45. I felt embarrassed during the simulation sessions
46. I was self-conscious of my performance during the simulation sessions/debriefing
47. I preferred having several simulations over one week
48. I preferred the timing of the sessions during July
49. I would have preferred longer sessions given during the first week of orientation
50. Free text: What did you find most useful during the sessions?
51. Free text: What did you like least about the sessions?
52. Free test: Suggestions for improvement of the simulation sessions

Abbreviation: CA-1, first-year clinical anesthesia resident.