Simulation based training versus non-simulation based training in anesthesiology: A meta-analysis of randomized controlled trials

Abstract

Background

Simulation based training (SBT) has become an important tool in medical education, yet a comprehensive review of its effect in anesthesiology is currently lacking. This study was aimed to compare the effect of SBT versus non-simulation based training (NSBT) in anesthesia education.

Methods

Data were derived from PubMed, Ovid (EMBASE and MEDLINE), China National Knowledge Infrastructure (CNKI) and China Biology Medicine (CBM) from database inception up to June 2022. We included published randomized controlled trials (RCTs) that assessed the impact of Simulation-Based Training (SBT) on anesthesiology training and compared it to Non-Simulation-Based Training (NSBT).

Results

25 eligible studies were included, with a total sample size of 1621 medical students. SBT was associated with improved theoretical scores [mean difference (MD):5.28, 95%CI 3.06 to 7.49, P < 0.01] and skill scores (MD: 7.63, 95% CI 5.7 to 9.57, P < 0.01) in comparison with NSBT. And SBT significantly improved teaching effects, including success rate, analytic skill in clinical setting, learning interest, learn efficiency and satisfaction survey. Also, a dramatic promotion was observed in the evaluation of non-technical ability consisting of cooperative ability, problem solving ability and situational awareness after trainees with SBT.

Conclusions

Simulation-based training is an effective method for improving theoretical scores, skill scores, teaching effects and non-technical ability. Yet, further good-quality RCTs are required to confirm these changes translate to improved patient outcomes. Furthermore, it is important to carefully consider the cost-benefit effect to conduct simulation-based in anesthesiology education.

1. Introduction

As an important aspect of clinical medicine, anesthesiologists are required to be capable of handling life-threatening events without making any mistakes [1]. Therefore, anesthesia education focuses on cultivating well-rounded clinicians who have a firm grasp of both theoretical knowledge and clinical skill [2]. Traditionally, non-simulation based training (NSBT) has been used to mainly teach theoretical knowledge through an apprenticeship model, such as didactic training, lecture-based training (LBL), problem-based learning (PBL) [3]. However, students under this type of training often lack sufficient experience in handling critical events and struggle with teamwork. Meanwhile, the tension of the doctor-patient relationship also adds difficulty to the clinical teaching. It is hard for medical students to combine theory and practice in clinical work.

To address this contradiction, anesthesia educators introduced simulation-based training (SBT) to imitate real-life clinical events in the late 1960s [4]. Since then, a wider application for simulation has been called for. Simulation teaching pioneers the use of full-body Mannikins, which can mimic some common skills, including tracheal intubation, nerve block, central venous catheter placement, etc [[5], [6], [7], [8]]. Additionally, SBT offers trainees opportunities to deal with rare conditions such as difficult airway management and malignant hyperthermia [3,9]. With the recent emphasis on crisis resource management (CRM), scenario simulation has become a hot topic. It is designed to target specific scenarios and foster technical and teamwork skills through role play within the context of those scenarios [10].

Although an increasing number of reports have indicated the efficacy of simulation in medical education [11,12], reviews comparing SBT to NSBT have been limited by nonsystematic identification of studies and a lack of quantitative synthesis. This study aimed to compare the effectiveness of SBT versus NSBT in anesthesia education and provide the latest evidence for anesthesia educational designs.

2. Methods

This study was planned, conducted and reported according to the recommended PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines [13].

2.1. Search strategy and criteria

With assistance from an experienced reference librarian, a systematic search of PubMed and Ovid (EMBASE and MEDLINE) was performed to identify published potential trails. Also, data was retrieved from two Chinese databases, including China National Knowledge Infrastructure (CNKI) and the China Biology Medicine (CBM). The search strategy is presented in Appendix 1. The last retrieval was performed on June 14, 2022.

The Population, Intervention, Comparison, Outcome, Study type approach as incorporated in the PRISMA Statement was used to define eligibility criteria to search for relevant literature. (1) The population consisted of students, residents, or workers within anesthesiology. (2) The intervention was featured SBT as any ‘tool with which the learner can physically interact in order to mimic a clinical scenario for purposes of training and/or assessment including computer-based simulators, mannikins, animal specimens, and cadavers, High-fidelity simulator’ [14]. (3) The comparator was NSBT defined as no technology-enhanced simulation. (4) Relevant studies were centered on the primary data of assessing learner and/or patient outcomes; (5) Study designs could be two-group randomized controlled trials (RCTs). Hand-searching was conducted to identify additional eligible studies from previously published review articles or google scholar.

The eligible included RCTs was assessed independently by two researchers (YS and YZ). Firstly, references were imported into EndNote X9 software (Thomson Reuters, New York, NY) to eliminate repeated literature. Then, the literatures were excluded according to information from title and abstract. Finally, full-text reading was screened for meta-analysis. Disagreements were reconciled through discussion among the team members.

2.2. Data extraction

The authors' information, publication date, sample size, anesthesia topic, characteristics of trainees, intervention, and outcomes were extracted. The primary outcomes consisted of scores on the theory and skill test. The secondary outcomes involved non-technical ability including cooperative ability, problem solving ability and situational awareness. Additionally, teaching effectiveness was evaluated for secondary outcomes, which included success rate, analytic skills in a clinical setting, learning interest, learning efficiency, ease of comprehension, and satisfaction with the training program.

2.3. Assessment of risk of bias

The methodological quality for the included RCTs was assessed independently by two researchers (YS and YZ) based on Cochrane risk-of-bias [15], and each quality item was graded as low risk, high risk, or unclear risk. The seven items used to evaluate bias in each trial included the randomization sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other bias.

2.4. Quality of evidence

The quality of evidence for each outcome was rated based on criteria established by the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) group. Quality of evidence was classified as very low, low, moderate, or high [15]. Any disagreement was settled through discussion among the research team.

2.5. Subgroup analysis

We planned subgroup analysis according to the following categories: 1) perioperative anesthesia: the whole operative anesthesia including preoperative assessment, plan formulation, anesthesia administration, postoperative resuscitation; 2) regional anesthesia: neuraxial or nerve blockade; 3) inhalation anesthesia: researches based on GasMan computer-based simulation; 4) CRM: studies that address CRM training.

2.6. Statistical analysis

Review Manager (RevMan for Windows, version 5.4; Cochrane Collaboration, Oxford, UK) was applied to perform all meta-analysis. For continuous variables, the data was expressed as mean difference (MD) or standardized mean differences (SMD). For dichotomous variables, the data was indicated as relative risk ratio (RR). The 95% confidence interval (95% CI) and point estimate were given for each outcome. I² test and chi-square test were used to assess heterogeneity (I² > 50% indicating high heterogeneity, I² < 50% indicating low heterogeneity) [16,17]. When P > 0.10 or I² < 50%, a fixed-effect model was used for combined analysis. Conversely, when P < 0.10 or I² > 50%, a random-effects model was used to pool the data, and we conducted a subgroup analysis to verify the outcome. And a sensitivity analysis was performed by leave-out method to identify the sources of heterogeneity. We assessed publication bias by examining funnel plots. All tests were 2-tailed, and P < 0.05 was statistically significant. The analysis plan was written before the data had been accessed. No significant deviated from the protocol during analyses procedure.

3. Results

3.1. Search results

1255 studies were preliminarily identified from Pubmed (1029), Ovid (38), CNKI (136) and CBM (52). 6 additional eligible studies were screened by Google Scholar. After duplicates were removed, 1176 studies were analyzed based on the title and abstract and 1134 studies were excluded. The remaining 42 articles were checked for eligibility based on full text reading and 17 studies being excluded, the 17 excluded studies consisted of five that had no training for anesthesiology, three lacking comparisons, four that were not randomized controlled trials, and five that had insufficient data (see details in Appendix 2). Ultimately, 25 articles [[5], [6], [7],9,[18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37]] met the inclusion criteria with a total of 1638 trainees. A Prisma-flow diagram of the literature search and the exclusion criteria is listed in Fig. 1.

Fig. 1.

Prisma-flow diagram for the literature search and exclusion criteria.

3.2. Study characteristics

The characteristics of the 25 studies included in this meta-analysis are listed in Table 1. These studies encompassed a total of 1638 trainees, with 819 trainees in the SBT group and 802 trainees in the NSBT group. The majority of trainees were interns, residents and junior anesthesiologists with limited related clinical experience. All the studies included in the analysis were published between 2005 and 2022. The sample size ranged from 10 to 108 trainees. Among these studies, 8 studies [3,5,9,[33], [34], [35], [36], [37]] were published in English and the other 17 studies [6,7,[18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]] were published in Chinese. The areas of anesthesia topics covered during the training were perioperative anesthesia (9 studies), airway management (3 studies), inhalation anesthesia (2 studies), regional anesthesia (2 studies), cardiopulmonary resuscitation (2 studies), CRM (2 studies). Each of the following subjects-anesthesia induction, cardiopulmonary bypass, transesophageal echocardiography, central venous catheterization, malignant hyperthermia-was the focus of one study. Outcomes reported in 15 studies reflected assessments in theoretical settings. And 12 studies were focused on assessments in clinical skills.

Table 1.

Demographic characteristics of the included studies.

Author/year	Sample size		Anesthesia		Intervention				Outcomes
	SBT	NSBT	topic	Trainees	SBT		NSBT
					Methods	Duration of the training	Methods	Duration of the training
Hall et al. 2005	18	18	AM	Intern	Manikin	10-h per trainee	RP	15 intubation attempts	SR
Wang et al. 2008	21	21	CPR	Intern	Manikin + PBL	6 CPR attempts +1 Manikin	PBL	6 CPR attempts	TT, ST, TEE (SS, AS, LI)
Hallikainen et al. 2009	23	18	AI	Intern	Manikin	1 week of lectures + 3 h instruction	RP	1 week of lectures + 3 h instruction	SR
Bruppacher et al. 2010	10	10	CPB	Junior	Manikin	1 week of lectures + 2 h instruction	ST	1 week of lectures + 2 h instruction	NTA(CA, PSA, SA)
Campos et al. 2011	13	14	AM	Postgraduate	Manikin	45-min per trainee	Video	14-min long video, maximum of two replays	SR
Ferrero et al. 2014	21	21	TEE	Residents	Manikin	45-min per trainee	DT	45-min per trainee	ST
Minai et al. 2014	28	29	AM	Intern	SS + Manikin	10–15 min per trainee	LBL + Observation	10–15 min per trainee	SR
Li et al. 2014	16	16	IA	Resident	CBS	6 lectures	LBL	6 lectures	TT
Lin et al. 2015	67	67	PA	Undergraduate	Manikin + CBL	Did not provide	CBL	Did not provide	TT, TEE (SS)
Wei et al. 2015	29	29	PA	Intern	SS + Manikin + CBL	Did not provide	CBL	Did not provide	TT, ST, TEE (SS)
Zou et al.2015	50	50	PA	Intern	HFS	20 lectures	DT	20 lectures	TT, ST
Gong et al. 2016	107	107	PA	Undergraduate	Manikin	Did not provide	DT	Did not provide	TT, ST
Guan et al. 2016	10	10	RA	Resident	Manikin	Did not provide	DT	Did not provide	TT, ST
Sheng et al. 2016	10	10	RA	Resident	Manikin + Multimedia	Did not provide	Multimedia	Did not provide	TT, ST
Wu et al. 2016	30	30	CVC	Undergraduate	Manikin + Multimedia	Did not provide	Multimedia	Did not provide	TT
Xie et al. 2016	10	10	CRM	Junior	Manikin + CBL	Did not provide	CBL	Did not provide	TT, ST
Ke et al. 2017	50	50	IA	Undergraduate	CBS + LBL	4 lectures	LBL	4 lectures	TT, TEE (AS, LI, LE, EU)
Zhao et al. 2017	20	20	PA	Intern	Manikin	Did not provide	LBL	Did not provide	TT, ST, TEE (SS, AS, LI, LE, EU)
Mejíaa et al. 2017	16	15	MH	Resident	HFS	A lecture and read 2 complementary literature reviews + one HFS lecture	CBL	A lecture and read 2 complementary literature reviews + one lecture	NTA(CA, PSA, SA)
Wei et al. 2018	19	19	CRM	Intern, Resident, Junior	HFS	4 lectures	PBL	4 lectures	ST, TEE (SS), NTA(CA)
Zhong et al. 2018	108	99	PA	Undergraduate	SS	Did not provide	DT	Did not provide	TT, ST, TEE (AS, LI, LE)
Berger et al. 2019	58	54	CPR	Intern	HFS	30-min per trainee	PBL	30-min per trainee	SR
Su et al. 2019	25	25	PA	Intern	HFS + CBL	Did not provide	DT	Did not provide	TT, ST, TEE (SS)
Wang et al. 2019	36	36	PA	Intern	SS	Did not provide	DT	Did not provide	TT, ST, TEE (SS)
Cao et al. 2020	24	24	PA	Resident	HFS	2-h theory + 2-h skill training	DT	2-h theory + 2-h skill training	SR, TT, NTA(PSA, SA)

Sample size/Intervention: SBT, simulation-based training; NSBT, non-simulation based training; Anesthesia topic: AM, airway management; CPR, cardiopulmonary resuscitation; AI, anesthesia induction; CPB, cardiopulmonary bypass; TEE, transesophageal echocardiography; IA, inhalation anesthesia; PA, perioperative anesthesia; RA, regional anesthesia; CVC, central venous catheterization; CRM, crisis resource management; MH, malignant hyperthermia; Trainees: Junior, junior anesthesiologist; Intervention: PBL, problem-based learning; SS, scenario simulation; CBS, computer-based simulation; LBL, lecture-based training; CBL, case-based learning; HFS, high-fidelity simulator; DT, didactic training; Outcomes: SR, success rate; TT, theory test; ST, skill test; TEE, teaching effect evaluation; SS, satisfaction survey for teaching; AS, analytic skill in clinical setting; LI, learning interest; LE, learning efficiency; EU, easy to understand; NTA, non-technical ability; CA, cooperative ability; PSA, problem solving ability; SA, situational awareness.

3.3. Risk of bias and quality of evidence

For the generation of random sequences, 16 studies describing the method was judged as low risks including random number table in 10 studies [3,5,8,22,23,31,[33], [34], [35],37] and random draw in 6 studies [6,21,24,26,28,30]. 3 studies randomly assigned with odd or even numbers were judged as high risks. The remaining 6 studies did not provide relevant information about the generation of random sequences and were judged as unclear risks. Only 3 studies described the method of allocation concealment, using sealed enveloping, and were classified as low risk. Other studies did not mention the allocation concealment method and were rated as unclear risk. Out of the total of nine studies, where blinding was implemented for participants and personnel, nine studies were categorized as low risk, as participants and implementers were not informed about the group assignment, while the remaining studies were evaluated as unclear risk, with no blinding information. However, 5 studies mentioning that independents assessed outcomes were classified as low risk. 5 studies were at high risks in which implementers took part in assessing outcomes and others were regarded as unclear risk without any information. The risk of bias item for each included study is shown in Supplementary Fig. 1. All studies were at low risk in incomplete outcome data, selective reporting, and other bias.

3.4. Meta-analysis

3.4.1. Theory test and skill performance

As shown in Fig. 2, 15 studies presented relevant data on theoretical scores (592 trainees in SBT and 583 trainees in NSBT). The pooled estimates of these studies showed significant improvement in SBT compared with NSBT (MD 5.28, 95%CI 3.06 to 7.49, P < 0.01, I² = 93%). GRADE evidence for theoretical scores within all included studies was moderate and downgraded for ‘inconsistency’ (Appendix 2).12 studies reported suitable data on skill test (445 trainees in SBT and 436 trainees in NSBT). SBT had favorable effect on skill scores in comparison with NSBT controls with (MD 7.63, 95% CI 5.7 to 9.57, P < 0.01, I² = 89%) (Fig. 2) GRADE evidence for skill scores within all included studies was moderate and downgraded for ‘inconsistency’ (Appendix 3). An assessment of the publication bias is shown in Supplementary Fig. 2 by using funnel plots of knowledge scores and skill scores, and no publication bias was found.

Fig. 2.

Forest plot of theoretical and skill test.

3.4.2. Subgroup analysis

Owing to the high heterogeneity in theory and skill test, we conducted sensitivity analysis. After eliminating studies one by one, the value of I² in theory and skill test was ranged 91%–93% and 70%–90%, respectively and the results of meta-analysis had no significant change. To further explore the sources of heterogeneity, we conducted subgroup analysis (Table 2). For theoretical scores, 8 studies [19,[23], [24], [25],28,[30], [31], [32]] included in perioperative anesthesia favored on SBT group compared with NSBT group and random-effects MD was 6.67 (95%CI 4.06 to 9.27, P < 0.01, I² = 94%). Simultaneously, sensitivity analysis showed no significant change in heterogeneity (I² ranged for 87%–95%). 2 studies [21,22] consisted in inhalation anesthesia showed a favorable effect on SBT in comparison with NSBT (MD 14.52, 95%CI 7.53 to 21.52, P < 0.01, I² = 54%). No significant difference was observed in the category of regional anesthesia [6,20] and random-effects MD was −2.27 (95%CI -5.11to 0.58, P = 0.08, I² = 0). The rest 3 studies [7,18,29] involved in airway management, central venous catheterization and CRM were excluded from subgroup analysis.

Table 2.

Results of subgroup analysis.

			No. of			Incidence			Subgroup difference
Outcomes	Subgroups	Studies	SBT	NSBT		MD	95%CI		Chi2	P	I2
Theory test	Perioperative anesthesia	8	442	443		6.67	[4.06,9.27]		120.48	<0.01	94%
	Inhalation anesthesia	2	66	66		14.52	[7.53,21.51]		2.17	<0.01	54%
	Regional anesthesia	2	20	20		−2.27	[-5.11,0.58]		0.80	0.12	0%
Skill test	Perioperative anesthesia	6	346	337		7.85	[7.18,8.52]		3.78	<0.01	0%
	Regional anesthesia	2	20	20		4.09	[1.71,6.47]		0.19	<0.01	0%
	Crisis resource management	2	29	29		9.35	[5.40,13.30]		0.05	<0.01	0%

No., number; SBT, simulation based training; NSBT, non-simulation based training, MD: Mean differences, CI: confidence interval.

In the subgroup analysis of skill test, 7 studies [19,24,25,28,[30], [31], [32]], Refs. [18,23,24,27,[29], [30], [31]] were classified into perioperative anesthesia had a favorable effect on SBT group (MD 7.11, 95%CI 4.19 to 9.31, P < 0.01, I² = 91%) and after eliminating one study [28], the heterogeneity was significantly reduced (MD 7.85, 95%CI 7.18 to 8.52, P < 0.01, I² = 0%). The subgroup of regional anesthesia [6,20] and CRM [27,29] included 2 studies respectively and trainees in SBT group showed higher scores than NSBT group. The meta-analysis showed MD of regional anesthesia was 4.09 (95%CI 1.71 to 6.47, P < 0.01, I² = 0%) and CRM was 9.53 (95%CI 5.4 to 13.3, P < 0.01, I² = 0%).

3.4.3. Evaluation of teaching effect and non-technical ability

The study used 6 items which evaluated teaching effect, including success rate, analytic skill in clinical setting, learning interest, learn efficiency, easy to understand and satisfaction survey. All evaluation items except easy to understand showed a favorable effect on SBT compared with NSBT (P < 0.05). Additionally, the evaluation of non-technical abilities, which included cooperative ability, problem-solving ability, and situational awareness, favored the SBT method over the NSBT method (P < 0.05). Table 3 presents the detailed results of the meta-analysis, which underwent sensitivity analysis by eliminating studies one by one.

Table 3.

Results of secondary outcomes analysis.

Secondary outcomes			No. of
		Studies	SBT	NSBT	RR (95%CI)		P	I2
Teaching effect evaluation	Success rate	6	433	425	1.64	[1.11, 2.43]	0.01	86%
	Analytic skill in clinical setting	4	101	101	1.95	[1.47, 2.59]	<0.01	22%
	Learning interest	4	101	101	2	[1.5, 2.56]	<0.01	30%
	Learning efficiency	4	188	179	1.76	[1.04, 2.99]	0.04	88%
	Easy to understand	2	70	70	2.45	[0.96, 6.26]	0.06	82%
	Satisfactory survey	5	177	177	1.47	[1.28, 1.68]	<0.01	0%
			No. of
Secondary outcomes		Studies	SBT	NSBT	SMD (95%CI)		P	I2
Non-technical ability	Cooperative ability	2	34	35	0.76	[0.27, 1.26]	<0.01	0%
	Problem solving ability	3	34	34	5.71	[4.58, 6.84]	<0.01	0%
	Situational awareness	3	34	34	5.88	[4.73, 7.04]	<0.01	0%

No., number; SBT, simulation based training; NSBT, non-simulation based training; RR: risk ratio, SMD: standardized mean differences, CI: confidence interval.

4. Discussion

Based on our findings in this systematic review of 25 studies, simulation training appears to be effective for acquiring knowledge and skills. However, the clinical or knowledge advantage remains a significant concern to justify the implementation of simulation in a medical program [38]. As indicated by a previous systematic review for anesthesia education, six studies (299 trainees) assessed skills and six studies (380 trainees) assessed knowledge both favoring simulation [39]. Similarly, in the current meta-analysis, we demonstrated the benefit of simulation education in theoretical knowledge acquisition and skill performance improvement. To exclude the influence of high heterogeneity on the results, we conducted sensitivity analysis by eliminating studies one by one, while the heterogeneity was still high. We then performed subgroup analysis, which showed that simulation teaching significantly improved the scores of theoretical and skill tests in perioperative anesthesia, inhalation anesthesia, regional anesthesia, and CRM. However, the subgroup analysis of regional anesthesia failed to demonstrate a significant benefit of SBT in theoretical knowledge acquisition. Only two studies with small sample sizes have reported on this field, which may help to explain the negative effect on theoretical knowledge acquisition among different anesthesia subjects [6,20]. Moreover, the tests used to evaluate theoretical knowledge acquisition are highly heterogeneous across studies, including differences in test content, training duration and so on. Thus, given these limitations, our results need to be interpreted with caution.

Anesthesiology is a subject with strong operational capability, and this traditional method is not enough for handling emergency situation. Therefore, simulation-based education has become a prevalent topic with offering an experiential learning paradigm for training and assessment, with opportunities to improve learning interest, communication, and decision-making skills for common as well as rare events [40]. Together with our study, we found all included studies showed a preferred effect on simulation teaching in comparison with no-simulation teaching including success rate, analytic skill in clinical setting, learn interesting, learn efficiency and satisfactory survey.

Anesthesiologists should possess high clinical skill, quick reaction ability and good teamwork ability to deal with CRM. Therefore, non-technical skills such as communication, team work, situation awareness, and decision-making are also indispensable skills for anesthesiology residents [41]. In our study, we found that trainees following simulation teaching showed a better capability in cooperative ability, problem solving ability and situational awareness. Admittedly, simulation teaching can mimic the clinical environment, the actual diagnosis, and treatment process, so that these trainees can fully grasp the key points and procedures of emergency anesthesia. As a result, the clinical teaching effect of simulation teaching is remarkable, and students can improve their clinical levels and adapt to clinical work more quickly.

5. Limitations

In this systematic review, we included only two-group (SBT versus NSBT) RCTs, which is both a strength and limitation [42]. In education, some have argued that RCTs have confused results, with possibly no practical application to the field of human affairs [43]. Then, we found high heterogeneity in most analyses, which was most likely caused by special requirements for schools and teachers to conduct simulation training. The different training levels of teachers and schools may have a significant impact on outcomes. Besides, anesthesiology is an integrated discipline involving perioperative anesthesia, regional anesthesia, CRM and some special items. One outcome measurement cannot cover all categories. Another important limitation to consider is the potential impact of trainee experience and training duration on the effectiveness of SBT and NSBT. Unfortunately, there is limited data available from primary studies that addresses this issue. Training duration varies greatly among studies and the experience level of training teachers is often unknown, which calls for caution when interpreting the conclusions of the current study. Besides, course content, the difficulty of examination, duration of courses, and preparation of teachers and students were different, which might have led to heterogeneity in the evaluation results. All of these may have contributed to the low score on GRADE. Finally, except for scores of theory and skill test, the questionnaire for teaching effect and non-technical ability is mainly derived from satisfaction survey, which lacked systematic and scientific evaluation indexes.

6. Suggestions for future study

From this system review, we have some suggestion for future study. Firstly, all literatures included in this meta-analysis can be classified into three Kirkpatrick levels: teaching effect for reaction (Kirkpatrick level 1), theory and skill test for learning (Kirkpatrick level 2), non-technical ability for behavior (Kirkpatrick level 3) [44]. However, no literature is included in level 4 (measured changes in patient outcomes) (Fig. 3). Without fully evaluation the patient outcome in anesthesia simulation research, it is unlikely that decision makers will devote the necessary resources and personnel toward this venture. Also, in these included trails, only 2 studies reported long-term retention including 5 weeks [34] and 6 months later [33]. Considering the different evaluation items, these results cannot be combined for quantitative analysis. Therefore, a more comprehensive and reasonable way to evaluate he potential advantages and benefits of simulation education in anesthesiology should be directed toward patient outcomes.

Fig. 3.

A summary for the benefit and cost of simulation training.

While we have objectively demonstrated that SBT is a promising method for anesthesia education, the huge financial and human resources should not be ignored (Fig. 3). First, feedback is the most important and frequently cited variable to assess the effectiveness of SBT [45]. Debriefing is the core and soul of simulation teaching feedback, while it’s a huge cost to establish different evidence-based model and sound recording system for variable disciplines. Moreover, not every teaching institution can afford to procure the necessary quantity of high-fidelity and multi-model mannequins required to replicate real clinical conditions. The instructors' role in facilitating, guiding and motivating learners is essential and there is a great need for a uniform mechanism to educate, evaluate and certify simulation instructors for the health care professions [46]. The implementation of simulation training for resource-limited regions and countries is limited. Deliberate practice is grounded in information processing and skill acquisition, while it’s hard to fill these gaps among sites, equipment and time for long-time deliberate practice. Educational experts have suggested that simulator practice should be carefully integrated with present educational events, including clinical experience, lectures and laboratory work [47]. The optimal approach to integrate SBT into existing curricula and the impact of this introduction on faculty and administrative resources remain to be explored. Although a growing number of studies are being performed to explore the mechanisms of skill decay, the results are inconsistent, and more research is clearly needed here. Additionally, we lack human resources for long-term follow-up evaluation in the current educational resources.

Overall, the results of this meta-analysis indicate that anesthesiology simulation-based training not only improves knowledge scores and skill scores but also enhances teaching effect and non-technical ability. However, it would be unwise to blindly expand applications, considering the significant resource investment it requires. There is a significant gap in outcome assessment, particularly regarding patient improvement. At a practical level, we should carefully balance the benefits and costs of SBT implementation in resource-limited regions and countries to meet the needs of social development for anesthesiology talents.

Author contribution statement

All authors listed have significantly contributed to the development and the writing of this article.

Data availability statement

Data included in article/supplementary material/referenced in article.

Authors' contributions

YS was responsible for the original study proposal. YS and YZ drafted the original study protocol and independently screened papers, extracted data and methodological quality. YS wrote the initial draft of the manuscript. YZ provided writing advice and methodological guidance. All the authors contributed to consecutive drafts and approved the final manuscript.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

Abbreviations

SBT

simulation-based training

NSBT

non-simulation based training

Anesthesia topic:AM

airway management

CPR

cardiopulmonary resuscitation

AI

anesthesia induction

CPB

cardiopulmonary bypass

TEE

transesophageal echocardiography

IA

inhalation anesthesia

PA

perioperative anesthesia

RA

regional anesthesia

CVC

central venous catheterization

CRM

crisis resource management

MH

malignant hyperthermia

Trainees:Junior

junior anesthesiologist

Intervention:PBL

problem-based learning

SS

scenario simulation

CBS

computer-based simulation

LBL

lecture-based training

CBL

case-based learning

HFS

high-fidelity simulator

Outcomes:SR

success rate

TT

theory test

ST

skill test

TEE

teaching effect evaluation

SS

satisfaction survey for teaching

AS

analytic skill in clinical setting

LI

learning interest

LE

learning efficiency

EU

easy to understand

NTA

non-technical ability

CA

cooperative ability

PSA

problem solving ability

SA

situational awareness

Appendix A. Supplementary data

The following is/are the supplementary data to this article: