Abstract
Background
The restrictions to stop COVID-19 pandemic have had a negative impact in simulation, however, it is imperative to develop new strategies that facilitate healthcare education.
Objective
To describe a simulation in healthcare based on the learning of Non-Technical Skills (NTS) and performed under the restrictions of COVID-19 Pandemic.
Methods
Quasi-experimental study of an educational activity performed through simulation with anaesthesiology residents in November 2020. Twelve residents participated in two consecutive days. A questionnaire was filled related to the performance of NTS that encompasses leadership, teamwork and decision making. The complexity of the scenarios and the NTS results obtained between the two days were analysed. Advantages and challenges were documented when a clinical simulation is performed under COVID-19 restrictions.
Results
The global performance of the teams improved when comparing first and second day (79.5% vs 88.6%, p < 0.01). Leadership was the worst section rated, however, was the one that showed the best improvement (70% vs 87.5%, p < 0.01). The complexity of the simulation cases had no relation with the group performance in leadership and teamwork but affected task management results. General satisfaction was over 75%. The main challenges to develop the activity were the technology required to adapt virtuality to simulation and the time spent for the preparation of it. No cases of COVID-19 were reported within the first month after the activity.
Conclusion
Clinical simulation can be done in the context of COVID-19 Pandemic, obtaining satisfactory learning results but requiring the adaptation of institutions to the new challenges it implies.
Keywords: Simulation training, Pandemics, COVID-19, Education
Abstract
Antecedentes
Las restricciones para frenar la pandemia de COVID-19 han tenido un impacto negativo en la simulación. Sin embargo, es imperativo desarrollar nuevas estrategias para facilitar la enseñanza sanitaria.
Objetivo
Describir una simulación sanitaria basada en el aprendizaje de competencias no técnicas (NTS) realizadas en el contexto de las restricciones de la pandemia de COVID-19.
Métodos
Estudio cuasi-experimental de una actividad educativa realizada mediante simulación con residentes de anestesiología en noviembre de 2020. Doce residentes participaron en dos días consecutivos. Se completó un cuestionario relativo al desempeño de NTS que abarcaba liderazgo, trabajo en equipo y toma de decisiones. Se analizaron la complejidad de los escenarios y los resultados de las NTS, obtenidos entre los dos días. Se documentaron las ventajas y dificultades de realizar una simulación clínica durante las restricciones de la COVID-19.
Resultados
El desempeño global de los equipos mejoró al comparar el primero y segundo días (79,5% vs 88,6%, p < 0,01). El liderazgo fue la sección peor calificada, aunque sin embargo fue la que reflejó mayor mejora (70% vs 87,5%, p < 0,01). La complejidad de los casos de simulación no guardó relación con el desempeño del grupo en cuanto a liderazgo y trabajo en equipo, pero afectó a los resultados sobre gestión de tareas. La satisfacción general fue superior al 75%. Las principales dificultades para desarrollar la actividad fueron la tecnología necesaria para adaptar la virtualidad a la simulación, y el tiempo empleado en su preparación. No se reportaron casos de COVID-19 durante el primer mes posterior a la actividad.
Conclusión
La simulación clínica puede realizarse dentro del contexto de la pandemia de COVID-19, obteniéndose resultados satisfactorios de aprendizaje, pero requiriendo la adaptación de las instituciones a las nuevas dificultades que ello implica.
Palabras clave: Formación en simulación, Pandemia, COVID-19, Educación
Introduction
Healthcare simulation (HS) is the process of recreating the characteristics of real clinical practice, allowing the trainer to carefully control the learning environment and optimise the conditions of the skill to be taught1. It is considered a safety strategy in the healthcare environment, which analyses where errors can be highlighted and corrected. This feedback process improves individual and collective knowledge, which can translate into better and safer medical practice2. HS has been compared to other learning strategies, showing positive results in the development of long-lasting knowledge and duration1, 3, 4, 5 and allowing medical schools to incorporate this strategy into their training programmes. For example, international educational guidelines in areas such as critical care, emergency medicine and anaesthesiology already include recommendations on the use of this strategy in their academic curricula6, 7, 8, 9.
HS has been negatively impacted by the COVID-1910 pandemic. It has been difficult to organise these activities in the context of the restrictions applied by the authorities11, 12. Furthermore, it is difficult to plan, organise and implement HS, considering that teachers and students are part of the healthcare personnel providing daily assistance to the pandemic, being potential victims, reservoirs and vectors of the virus10. However, further training and updating of healthcare personnel is essential, especially in the current context of overcrowded facilities, where technical and non-technical skills (TNTS) are necessary to provide qualified healthcare to patients13. With these considerations in mind, HS must adapt to new biosafety measures to continue to play an important role in the context of medical education and healthcare.
Our objective is to describe an HS activity with anaesthesiology resident physicians focusing on the development of TNTS and conducted under COVID-19 pandemic biosecurity measures.
Materials and methods
A quasi-experimental pre-post design study was conducted with 12 anaesthesiology residents in their second to fourth year of training at the Vall d'Hebron University Hospital in Barcelona (Spain) in November 2020. Inclusion was voluntary and in keeping with personal interests, following a call explaining the initiative. Thus, 5 second-year, 3 third-year and 4 fourth-year residents were included in the study, out of a total of 46 residents. In addition, 3 external observers, 5 peers and 2 moderators participated in the follow-up and discussion of the cases. The intervention was divided into 2 phases. The first consisted of theoretical content on the acquisition of knowledge about critical anaesthesia situations and the creation of the simulated cases. The second phase corresponded to the practical scenarios in which the HS took place.
Theoretical learning phase and setting design
The participants were divided into groups of 2 persons, with each group being assigned a specific subject. These subjects, provided by the moderators, were related to critical situations in anaesthesiology such as bronchospasm, anaphylaxis, massive haemorrhage, difficult airway management, cardiorespiratory arrest, local anaesthetic toxicity, malignant hyperthermia and polytrauma patients. The first part of the training consisted of a general online session for all participants, which included an introductory video on basic simulation concepts and clinical scenario design. This was followed by a 10 h online session for basic simulation instructors. The course content was divided into 5 modules, and the subjects were as follows: a) HS; b) Teaching simulation competencies; c) TNTS; d) Designing simulation scenarios; and e) Debriefing.
A period of one month was granted for the preparation of a literature review, specific to the previously assigned case. The residents had to share it with their colleagues on a collaborative learning basis, after submission and supervision by the moderators. In addition, they constructed simulation cases based on the assigned subject matter, using the model proposed by the Centre for Advanced Clinical Simulation of Vall d'Hebron. This was done to facilitate learning simulation techniques and to familiarise them with the practical activity to be performed, as they had limited experience of HS. Supervision of these activities was carried out via email.
Clinical simulation settings practice
The simulation scenarios were performed in the operating room of the outpatient surgery facilities of the Vall d'Hebron University Hospital, during 2 consecutive days. A high-fidelity manikin and standardised patients were used, depending on the cases. Participants were divided into groups of 3 residents. Each team performed 2 simulated scenarios. Due to capacity limitation regulations, a maximum of 6 persons participated in each scenario, including residents, peers and mediators. The scenarios lasted about 15 min, and the subject matter was different from that assigned in the theoretical part, so the residents were kept in the dark about the next simulation case.
The moderators then classified the simulation cases as low, intermediate or high complexity. Classification criteria included aspects such as distracting factors, cognitive overload, resource limitation and unexpected turn of events. Low complexity simulation cases involved more linear developments, with no distracting factors or physical and cognitive resources available to the participants. High complexity simulation cases corresponded to those that generated significant cognitive overload for participants, with limited available resources, confounding factors within the scenario and an unexpected and rapid turn of events.
During the simulation, one team played the clinical scenario and the rest of the groups were spectators. The rest of the participants, including peers, other residents and external observers, followed the live stream telematically via Microsoft Teams®. After each case, a TNTS-based debriefing session was held, with all residents and external observers following the webcast being able to participate.
Attendees of the live practice had to complete an epidemiological survey on COVID-19. The HS was conducted according to the biosafety measures implemented by the authorities, using masks, gloves and hand washing before and after the cases. In addition, each group used the simulation scenario, and all surfaces were cleaned by the participants so that the next group could perform their own case. After one month, each participant was contacted directly to assess whether they had had COVID-19 infection within that period.
Following the case simulation, the moderators discussed the strengths and challenges identified during the process. The information was compiled into a report, which was used to describe this item in the following results.
Instruments of measurement
Two assessment tools were used. The first was the Team Emergency Assessment Measure (TEAM) scale. This is a globally validated scale for assessing TNTS in critical situations13 which has a Spanish version (Appendix B Annex 1). It consists of 11 items grouped into 3 sections: a) leadership; b) teamwork; and c) task management. Each item has a score ranging from 0 to 4 (never/almost never, rarely, often, often, and always/almost always, respectively). It also has an overall rating scale at the end of the questionnaire to assess overall performance, ranging from 1 to 10. This form was completed by all participants, including residents, moderators, external observers and peers, at the end of each scenario. The aim of this evaluation was to obtain an overall perspective of the teams' performance. It was also intended to enhance the experiential learning process of the participants through self- and peer-assessment.
The second evaluation tool consisted of a satisfaction survey completed by the residents at the end of the training session, which was created by the researchers and submitted for peer review. It was designed using a self-administered questionnaire with a Likert-type score from 0 to 5, where the items assessed included perception of the learning experience, evaluation of the theoretical and practical phases, and overall satisfaction with the activity.
Ethical considerations
This study was approved by the Drug Research Ethics Committee and the Research Projects Committee of the Vall d'Hebron University Hospital (PR [AG] 448/2020). The certificate was issued on 18 September 2020.
Statistical analysis
Statistical analysis was performed using the IBM-SPHS® version 22 software. We used the calculation of means and medians as measures of central tendency for the overall performance and by sections of the TEAM scale, using the standard deviation and interquartile range as measures of dispersion. In order to be able to analyse the overall performance, we added the results of the 3 sections of the TEAM scale. This allows us to give scores from 0 to 44 for each evaluation form, as each section has a range between 0 and 4, and the tool has 11 items. An inferential analysis was carried out, comparing the measures obtained from the simulation cases on the first and second days. As the variables did not follow a normal distribution, the Mann-Whitney U test was used to contrast quantitative variables with dichotomous qualitative variables, and the Kruskal-Wallis test was used to contrast quantitative variables with non-dichotomous qualitative variables. Values of p < .05 were considered significant.
Results
Twelve residents participated in the HS intervention, which was organised in 4 groups of 3 persons each. A total of 8 simulation cases (2 cases per group) were developed during the 2-day practical activity. A total of 130 TEAM forms were completed: 66 for the first day and 64 for the second day. However, 12 forms were discarded due to incomplete data, resulting in a net balance of 118 valid questionnaires.
Results of the TEAM scale
The TEAM scale scores of the first- and second-day cases were compared. The median value during the first day was 35 (IQR 29.75–41), representing an overall performance of 79.5%, and which during the second day was 39 (IQR 36–42), representing an overall performance of 88.6% (p < .01). Differences in TEAM scale scores were obtained according to the complexity of the simulation cases. For low complexity cases, the mean value was 39 (IQR 33−42); for intermediate complexity cases, the mean score was 38 (IQR 32.5−41), and for high complexity cases, the mean value was 33 (IQR 28–38).
The degree of complexity of the cases was compared between the first and second day of the activity. On the first day, 53% of cases were of low complexity, 24.2% of cases were of intermediate complexity, and 22.7% of cases were of high complexity. On the second day, 30.8% of cases were of low complexity and 69.2% of cases of intermediate complexity, with no cases of high complexity on that day. The difference in complexity of the simulation cases in relation to the day reached statistical significance (p < 01). However, when analysing the results by sections of the TEAM scale, there were no differences related to the degree of complexity, with the exception of the task management section (Table 1 ). In contrast, better performance was documented in the leadership and teamwork sections on the second day of the activity (Table 2 ).
Table 1.
Results from the sections of the TEAM scale with regard to case complexity.
| Low grade | Intermediate grade | High grade | p | |
|---|---|---|---|---|
| Leadership | 3 (2.5−4) | 3.5 (2.6−4) | 3 (2.5−3.5) | .10 |
| Teamwork | 3.6 (3.1−3.9) | 3.6 (3−3.9) | 3.3 (2.9−3.7) | .14 |
| Task management | 3.5 (3−4) | 3.3 (3−4) | 2.5 (2.5−3) | <.01 |
Data are presented as median (interquartile range).
Table 2.
Comparison between TEAM scale results by sections, with regard to each day of simulation.
| Day one | Day two | p | |
|---|---|---|---|
| Leadership | 2.8 (2−3.5) | 3.5 (3−4) | <.01 |
| Teamwork | 3.3 (2.9−3.9) | 3.6 (3.3−3.9) | .01 |
| Task management | 3 (2.5−4) | 3.5 (3−4) | .03 |
Data are presented as median (interquartile range).
Results from the satisfaction survey
When asked about the overall satisfaction with the simulation activity, 75% of the participants rated it between 4 and 5 on a range of 0 to 5. All participants responded that they had acquired new TNTS that could be useful for their daily clinical activity. Furthermore, 87.5% of the residents would repeat the activity, and 100% would recommend it to other colleagues. For 33.2% of the participants, this activity represented a workload for their daily duties. One of the participants stated that there was organisational disorder between switching from one team to another during the simulation cases, which is an area for improvement. Another participant reported difficulties with the internet connection for the live streaming of the activity.
No cases of COVID-19 were reported among the participants one month after the activity.
Discussion
The objective of this project was to evaluate the possibility of conducting a clinical simulation activity based on the development of TNTS with current biosecurity measures to curb the COVID-19 pandemic. Based on the results of TEAM, an improvement in the performance of the participants was observed throughout the evolution of the training session. This tool demonstrated its reliability in measuring staff performance in critical situations, as well as its ease of implementation, as it can be used by inexperienced staff without incurring measurement error14, 15. The data obtained in this study reflect similar results to those of other authors in the field of HS, or real-life clinical practice16, 17. The leadership section scored the lowest in the simulations; however, it was also the section that showed the greatest improvement after the training sessions. These findings are consistent with the results obtained by Cooper et al. who evaluated the TEAM scale in the context of cardiorespiratory arrest16.
It is important to note that the overall results of the TEAM scale were influenced by case complexity. However, analysing by subgroups, a statistically significant relationship was only achieved when comparing the complexity of the clinical scenario with the task management section, so it can be concluded that the higher the complexity of the case, the lower the performance in this section. In contrast, no such association was observed in the leadership and teamwork sections. However, an improvement in the performance of the groups in these 2 sections could be documented, comparing the results between the second and the first day of training. It should also be noted that, based on the percentile scores obtained in our study, and in contrast to the available literature, overall performance can be considered poor with 33 points or less, good with scores between 34 and 39, and excellent with scores above 4016.
Based on the experience of a HS conducted during the COVID-19 constraints, we documented several difficulties related to the logistics of the activity. The first was the need for technology to facilitate the sessions. It was necessary to adapt the operating theatre area with cameras, microphones and internet connection to provide a clear signal for transmission to the telematically connected participants. In addition, a room with computers, loudspeakers and microphones was adapted for the residents who had to participate in the following simulation cases. This difficulty had already been expressed by other authors regarding the need for technology as a didactic tool12, 17. In fact, organisers should be aware of the difficulties with internet connection, software for virtual platforms, and signal quality, elements that could compromise the quality of the training session. This consideration is even greater in low-budget environments, where access and availability of technology is difficult. It has been reported that in developing countries the effectiveness of e-learning may be reduced due to the lack of the necessary infrastructure in schools, or due to students' limitations in acquiring the appropriate devices18. Another potential obstacle to this strategy includes the risk of breaching the confidentiality and security of students, as sessions are transmitted over the Internet. This concern was also expressed in a literature review conducted by Wilcha17. In addition, the preparation and organisation of the event required the use of additional time for the mediators, who were also trained in the use of new technologies. This limitation was also reported by Gaur et al. in relation to pre-clinical teaching during the COVID-19 pandemic11.
In contrast to the difficulties mentioned above, this activity performed well in terms of the overall satisfaction of the participants, with a 75%–100% response to the survey. Participants stated that they had acquired new skills for their professional practice, and that they would recommend the activity to other colleagues. This subjective improvement could be corroborated by the development of the TEAM scale explained in previous sections. However, it is important to mention that a third team felt that the performance of the tasks was too much of a workload and was a source of discomfort.
Several limitations must be acknowledged. Firstly, the number of participants was small, representing 26% of the total number of residents enrolled in our anaesthesiology programme. Their participation was voluntary, which may imply an undercurrent of special motivation, which may affect the satisfaction results. Secondly, as this was not a randomised sample, it has inherent biases. Thirdly, when participants completed the TEAM scale, they did not describe the role they played in the simulation cases (residents, external observers, moderators or peers). It would have been interesting to identify whether there were differences between the assessment and the role played. However, the aim of this study was to have a 360-degree view of the performance of TNTS, and for the participants to get an overall impression of it. Finally, we assessed COVID-19 infection in the participants by questionnaire one month after the activity. However, none of them had clinical evidence to ensure that they were not asymptomatic cases, which is another limitation to consider.
Conclusions
Health education should be the cornerstone of patient care, and clinical simulation is a useful tool. Despite current restrictions due to the COVID-19 pandemic, it can be performed safely and has positive results when it comes to acquiring new knowledge or skills. For this to be possible, biosafety measures and new technologies must be incorporated, knowing that this implies additional effort and difficulties for educators and institutions. This must be addressed to ensure a quality training experience.
Funding
The authors declare that this study did not receive any type of funding.
Authorship
The authors contributed equally to this document and read and approved its final version.
Conflict of interests
The authors have no conflict of interests to declare with any financial entity with regards to this document material.
Acknowledgements
The authors wish to thank doctors F. Munar, P. Tormos and M. Pascual for their cooperation throughout this study.
Footnotes
Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.redare.2021.08.001.
Appendix A. Supplementary data
The following is Supplementary data to this article:
References
- 1.Beal M., Kinnear J., Anderson C.R., Martin T.D., Wamboldt R., Hooper L. The effectiveness of medical simulation in teaching medical students critical care medicine: A systematic review and meta-analysis. Simul Healthc. 2017;12:104–116. doi: 10.1097/SIH.0000000000000189. [DOI] [PubMed] [Google Scholar]
- 2.Ferrero F. ¿Puede la simulación clínica contribuir al aprendizaje significativo de competencias educativas? Una aproximación constructivista. Rev Fac Med UNAM. 2017;60(Supl 1):46–59. [Google Scholar]
- 3.Cooper S., Cant R., Porter J., Bogossian F., McKenna L., Brady S., Fox-Young S. Simulation based learning in midwifery education: A systematic review. Women Birth. 2012;25:64–78. doi: 10.1016/j.wombi.2011.03.004. [DOI] [PubMed] [Google Scholar]
- 4.Lorello G., Cook D., Johnson R., Brydges R. Simulation-based training in anaesthesiology: A systematic review and meta-analysis. Br J Anaesth. 2014;112:231–245. doi: 10.1093/bja/aet414. [DOI] [PubMed] [Google Scholar]
- 5.Morse C., Fey M., Kardong-Edgren S., Mullen A., Barlow M., Barwick S. The Changing landscape of simulation-based education. Am J Nurs. 2019;119:42–48. doi: 10.1097/01.NAJ.0000577436.23986.81. [DOI] [PubMed] [Google Scholar]
- 6.Fuentes Amaya C.M., Corvetto Aqueveque M.A., Altermatt Couratier F.R., Delfino Yurín A.E. Incorporación de la simulación en el entrenamiento del manejo de recursos en crisis, experiencia del programa de anestesiología de la Pontificia Universidad Católica de Chile. ARS Média. 2017;42:27–133. [Google Scholar]
- 7.Chiu M., Tarshis J., Antoniou A., Bosma T.L., Burjorjee J.E., Cowie N., et al. Simulation-based assessment of anesthesiology residents’ competence: Development and implementation of the Canadian National Anesthesiology Simulation Curriculum (CanNASC) Can J Anaesth. 2016;63:1357–1363. doi: 10.1007/s12630-016-0733-8. [DOI] [PubMed] [Google Scholar]
- 8.L’Her E., Geeraerts T., Desclefs J.P., Benhamou D., Blanié A., Cerf C. Simulation-based teaching in critical care, anaesthesia and emergency medicine. Anaesth Crit Care Pain Med. 2020;39:311–326. doi: 10.1016/j.accpm.2020.03.010. [DOI] [PubMed] [Google Scholar]
- 9.Orden SSI/81/2017, de 19 de enero, por la que se publica el Acuerdo de la Comisión de Recursos Humanos del Sistema Nacional de Salud, por el que se aprueba el protocolo mediante el que se determinan pautas básicas destinadas a asegurar y proteger el derecho a la intimidad del paciente por los alumnos y residentes en Ciencias de la Salud. BOE núm. 31, de 6 de febrero de 2017. pp. 8277-89.
- 10.Tabatabai S. COVID-19 impact and virtual medical education. J Adv Med Educ Prof. 2020;8:140–143. doi: 10.30476/jamp.2020.86070.1213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gaur U., Majumder M.A.A., Sa B., Sarkar S., Williams A., Singh K. Challenges and opportunities of preclinical medical education: COVID-19 crisis and beyond. SN Compr Clin Med. 2020;2:1192–1197. doi: 10.1007/s42399-020-00528-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tisi Lanchares J.P. Clinical simulation after COVID-19 pandemic. Int J Dental Sc. 2020;22:16–18. [Google Scholar]
- 13.Okuda Y., Bryson E.O., DeMaria S., Jr., Jacobson L., Quinones J., Shen B., Levine A.I. The utility of simulation in medical education: What is the evidence? Mt Sinai J Med. 2009;76:330–343. doi: 10.1002/msj.20127. [DOI] [PubMed] [Google Scholar]
- 14.Boet S., Etherington N., Larrigan S., Yin L., Khan H., Sullivan K. Measuring the teamwork performance of teams in crisis situations: A systematic review of assessment tools and their measurement properties. BMJ Qual Saf. 2019;28:327–337. doi: 10.1136/bmjqs-2018-008260. [DOI] [PubMed] [Google Scholar]
- 15.Freytag J., Stroben F., Hautz W.E., Schauber S.K., Kämmer J.E. Rating the quality of teamwork-a comparison of novice and expert ratings using the Team Emergency Assessment Measure (TEAM) in simulated emergencies. Scand J Trauma Resusc Emerg Med. 2019;27:12. doi: 10.1186/s13049-019-0591-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Cooper S., Cant R., Connell C., Sims L., Porter J.E., Symmons M., Nestel D., Liaw S.Y. Measuring teamwork performance: Validity testing of the Team Emergency Assessment Measure (TEAM) with clinical resuscitation teams. Resuscitation. 2016;101:97–101. doi: 10.1016/j.resuscitation.2016.01.026. [DOI] [PubMed] [Google Scholar]
- 17.Wilcha R.J. Effectiveness of virtual medical teaching during the COVID-19 crisis: Systematic ceview. JMIR Med Educ. 2020;6 doi: 10.2196/20963. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Carvalho V.O., Conceição L.S.R., Gois M.B. COVID-19 pandemic: Beyond medical education in Brazil. J Card Surg. 2020;35:1170–1171. doi: 10.1111/jocs.14646. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.