Abstract
Rising concern about patient safety has resulted in growing interest in non-technical skills (NTS) among anaesthesiologists. Growing evidence suggesting the use of good NTS training in patient safety in simulated as well as real-world environment made them important in medical education. Both technical skills (TS) and NTS are interdependent. Successful task performance depends on effective integration of both TS and NTS for any given situation. Development of tools for assessing the NTS of an anaesthesiologist in improving health care outcomes is challenging. Teaching, understanding and evaluating NTS among anaesthesiologists in improving health care outcomes is a domain which is supposed to be a rich seam for future studies.
Keywords: Assessment, curriculum, non-technical skills
INTRODUCTION
In recent times, greater accountability in professional performance is required from anaesthesiologists for patient safety. The post graduate (PG) curriculum in anaesthesia focuses largely on acquiring knowledge and clinical skills required for clinical practice. Although technical skills (TS) are essential for effective performance in the operating room (OR), lack of non-technical skills (NTS) has consistently been responsible for a majority of errors and adverse patient outcomes in the peri-operative period. NTS are the cognitive, social and personal resource skills, which complement TS. NTS contribute to safe and efficient task performance of team members, which in turn helps in avoiding errors that occur due to human factor issues.
NTS are broadly divided into two groups. Cognitive and mental skills such as decision making and situation awareness (SA), which are important at an individual level, and social and interpersonal skills such as teamwork, communication and leadership, which are important at a team level.
NTS were incorporated for the first time in commercial aviation initially in the cockpit, later on in the crew resource management (CRM, 1980) and the non-technical assessment in aviation (‘NOTECHS’ 1990). The anaesthesia crisis resource management (ACRM) framework was introduced in 1990 by Howard et al.[1] The ACRM framework is based on key elements like communication, situation awareness, decision making, task management, teamwork and leadership.
Importance of NTS to the anaesthesiologist
The PG curriculum in anaesthesia focuses on TS needed for independent anaesthesia practice. However, up to 80% of all errors in medicine can be attributed to the human errors and unawareness of NTS.[2] Relationship between TS and NTS was investigated in a study in Denmark.[3] This study illustrated the intertwined relationship between TS and NTS and the impact of this relationship on patient management. An effective integration of TS and NTS is needed for successful task management and prevention of adverse events in healthcare. The NTS are an important part of competent anaesthesia practice.[2]
Human factors are the study of environmental, technical, organisational, psychological and physiological influences on human performance. Jones et al.[4] reported the importance of human factors in clinical practice. In the high-risk accident-prone environment of the OR and the emergency department, the absence of NTS resulted in negative outcomes. The fourth national audit project (NAP4) analysed major complications related to airway management during anaesthesia. In the 184 reports reviewed, human factors were highlighted as contributors for the complications that occurred in most cases (median range 4.5 per case).[5] In the 5th national audit project (NAP5), which reviewed accidental awareness during general anaesthesia, 73% of incidences of awareness under anaesthesia were reported due to miscommunication.[4] The importance of NTS in reducing medical errors related to human factors has been recently recognised.
Various NTS required for anaesthesia practice, their relevance and role in enhancing patient safety and clinical outcomes are detailed as follows.
NTS framework for anaesthesiologists
TASK MANAGEMENT
Lack of management skills was identified as a major contributing factor for poor and unsafe care towards the patients and can lead to the collapse of teamwork, which is the most required in an OR. In addition to anaesthetic competence, successful task management depends on the effective integration of TS and NTS for any given situation. Task management comprises of the following:
Planning and preparation
This involves developing primary and contingency strategies in advance in order to handle tasks. These must be regularly reviewed and appropriately enhanced to reach the set goals. Advance preparation and planning for drugs, required equipment and strategies before starting a case exhibits an excellent behavioural marker for an anaesthesiologist.
Prioritising
The OR is nothing less than a war front, involving both triage and scheduling, with a tough tag ‘prioritise’. Whether an elective or an emergency list, cases need to be prioritised as needed. Lack of prioritising will lead to failure of allocating resources to critical areas.
Providing and maintaining standards
An anaesthesiologist is expected to adhere to accepted anaesthesia principles by following universal standards of good practices, treatment protocols and guidelines. Cross-checking drug labels, checking equipment and maintaining records are part of good practices.
Identifying and utilising resources
People, equipment, time, expertise and drugs need to be identified, planned, segregated and allocated to places as required. This helps to create a stress-free environment. Acquiring and refining management skills is an individual and ongoing process.[6] Therefore, this skill needs to be assessed and re-evaluated, focusing on individual components during PG anaesthesia training.
TEAMWORK
Teamwork is a skill that needs to be developed to work in a group for effective joint task completion and team member satisfaction. This needs coordination with team members, exchange of information, use of authority and assertiveness and assessment of the capabilities of the team members.
Both the surgeon and the anaesthesiologist work as a team, jointly caring for the patient along with the nursing staff and the technical staff in the OR. Everyone has a common role in achieving good outcome for the patient. Each one has their own command, hierarchy and own set of local goals and objectives for caring for a patient. The quality of teamwork directly influences clinical performance in anaesthesia.[7]
As a part of team work, it is very important to coordinate activities with the rest of the team members. The pre-operative period is very crucial as major decision making for the anaesthetic plan is done during this period. Effective communication between the surgical, anaesthesia and other OR team members will help the anaesthesiologist to design an anaesthetic plan for better patient outcomes. The surgical team should be alerted by the anaesthesiologists regarding any untoward event during the surgical procedure. Communication should remain open on both sides and at all the times. Every team member is contributing to patient care. Assessing capabilities and skills of team members will help in improving the team dynamics and patient safety.
SITUATION AWARENESS
The concept of situation awareness (SA) was introduced into the field of anaesthesia in 1995 by Gaba et al.[8] SA can be applied at an individual level (personal SA) and also at the team level (shared SA). SA describes the ability of an individual to maintain an adequate internal representation of the status of the environment in a complex and dynamic situation, where conditions may change within seconds and minutes.[9] Team SA is defined as the degree to which every team member possesses the SA required for their responsibilities.
SA as an important component of anaesthetists’ non-technical skills (ANTS) comprises of perception, comprehension, projection and decision-making.
Perception
This involves gathering the information and data from the team to identify the problem. Many sources of information need to be scanned to get an overview of the situation. This can be explained by taking an example of haemodynamic instability. If it occurs intra-operatively, the anaesthesiologist needs to evaluate the situation by assessing the trend in blood pressure, fluid administered and blood loss during the surgical procedure.
Comprehension
This involves recognising, understanding and interpreting the information. It is important to find out the reason for haemodynamic instability, the response to the medications given and assess the further need for fluid and blood transfusion.
Projection
This involves anticipating the seriousness of the situation with proper actions to mitigate potential problems. The anaesthesiologist may need to completely take charge of the situation, including arranging for more blood and blood products if required. Awareness of maintenance of sufficient perfusion pressure is necessary during such crisis.
Success or failure of SA can occur at the level of each component. To determine the frequency of SA errors, 200 cases from the German anaesthesia critical incident reporting system were reviewed.[10] SA errors were identified in 81.5% cases. Errors occurred predominantly at the level of perception (38%) and comprehension (31.5%).
DECISION-MAKING
Decision-making is a cognitive process, which involves identifying options to take a proper decision, balancing risks and selecting options with proper implementation and re-evaluation of the cases and situation. Expert and accurate decision-making is critical for the patient safety. Decision-making requires an individual to develop skills that will help in taking proper action in a crisis situation. Decision-making involves skills for making a judgment to select a course of action in both normal as well as time-pressured crisis situations. Peri-operative dynamic decision-making as an important component of the cognitive process is required to be undertaken by an anaesthesiologist. In routine clinical practice in the OR, anaesthesiologists are faced with several decision points.[11] Practising safe anaesthesia involves consistently making ‘good’ decisions. Decisions taken by the anaesthesiologist may occur over a period of days, often beginning with pre-operative assessment.
Human factors at the team level
COMMUNICATION
Patient safety and good communication are prerequisites of good peri-operative care. Basic communication skills are a standard of professional cognitive competency in clinical practice. Communication breakdown among clinicians, patients and families may lead to medical errors that may be prevented through effective communication. Effective communication is the most important component of NTS. Studies have estimated that errors related to miscommunication have led to as high as 251,000 deaths in the USA.[12] Good communication practices create a trustworthy relationship, instil confidence, enhance patient safety and create a positive impression about the anaesthesiologist and the work culture. Anaesthesia care is divided into three distinct phases: pre-operative evaluation, peri-procedural management and post-anaesthesia care. Each phase poses unique communication challenges, with different demands. Due to their focus on peri-operative patient care and safety issues, effective communication with patients and families often takes a back seat among anaesthesiologists. This further contributes to the lack of awareness of the anaesthesiology speciality among patients and their families.[13] They often imagine the anaesthesiologist to be a ‘behind the screen physician’ whose job only consists of administering medications to put patients to sleep before an operation.
Poor communication with patients and their families has been shown to lead to higher anxiety levels and lower satisfaction levels. Anxiety may increase cortisol secretion, considered as an acute stressor and has been observed among patients awaiting surgery.[14] A recent randomised trial using pre-operative interactive communication with anaesthesiologists via an anaesthesia service platform before laparoscopic cholecystectomy in 222 female patients, showed a reduction in anxiety levels and faster rehabilitation.[15]
Effective communication prior to surgery has been shown to be a good way to earn the patient's trust, increase co-operation during treatment, decrease anxiety and pain levels, reduce complications and increase confidence in fighting the illness. Although pamphlets and audio-video messages are often used for pre-operative education, studies have shown that these methods do not allow patients to have interactive communication with their anaesthesiologist and patients may lack the ability to understand them completely.[16]
A scoping review of 25 studies conducted in 27 countries across six continents of in-hospital adverse events showed that poor communication is the cause of a majority of the adverse events.[17] Studies have shown that while there is good communication among surgical colleagues, between surgeons and nurses, only 29% anaesthesiologists have meaningful communication with surgeons.[18] Effective communication with the surgeons and other co-workers throughout the patient's peri-operative journey can help in patient optimisation for surgery, planning the anaesthesia, intra-operative and post-operative management including handling unanticipated complications that may occur.
Formal attempts proposed to improve communication in the OR and hence patient safety through the use of the surgical checklist[19] and practising a ‘time out’ should be encouraged.[20] Effective communication during an emergency situation is vital to minimise errors, avoid complications and tide over the crisis effectively.[21] Example of such situations are failed upper airway management, haemodynamic collapse or cardiac arrest. Clear and uniform ‘critical language’ should be used in such situations to aid communication, understand the nature of the emergency, avoid confusion and improve team situational awareness. Effective communication is an art, an essential skill for the anaesthesiologist that can be acquired through observation of peers and practice.
LEADERSHIP
Leadership is the capacity of a leader to translate the vision into reality. Good leadership makes a difference in institutional progress.[22] Institutions need medical leadership with not only good knowledge and clinical skills but also with competencies such as time management, communication, organisational and business skills [Table 1]. Four important competencies (knowledge, skill, attitude and vision) are required for leadership.[23]
Table 1.
Be fair, honest and trustworthy. |
Be democratic. |
Be silent about your sacrifices. |
Do not reveal your weaknesses. |
Be critical thinkers. |
Be intelligent, highly analytical and rational decision makers. |
Convey information effectively. |
Be focused and results oriented. |
Prioritise your tasks to manage time. |
Command responsibility. |
Self-awareness (reach the highest pinnacle). |
Knowledge competencies
Knowledge of the team mission and objectives should be clear and well understood by the leader. To assign task-specific responsibilities, the leader needs to understand each team member's characteristics, strengths, weaknesses and abilities.
Skill competencies
Situation analysis and decision making are the core skills required for a successful outcome. A leader should have the ability to recognise obstacles and adapt to changes needed in teammates and strategies to accomplish the mission.
Emotional intelligence (EI)/attitude competencies
The leader should have positive vibes to inspire others to believe in a team approach and mutual trust.
Vision
Having a good vision is the first step towards achieving a goal.
EMOTIONAL INTELLIGENCE
EI is an emerging novel psychological construct in scholastic, academic, educational and organisational settings that forms an integral component of the NTS. Incorporation of EI has emerged as a major contributor to the safe and efficient task performance.[24] There are four integral components encompassing the entire concept of EI.
Self-awareness is the first important component of EI that primarily readdresses the importance of being aware of one's own potential to incorporate the feedback from a critique or criticism, as well as the capability to distance from certain situations aptly while not allowing oneself to indulge in bias and prejudice.[25] Self-regulation, the second integral component, is a skill to be mastered to control certain emotions under crisis situations. Most important is controlling impulsivity, anger and pessimism that may hamper the outcomes. The third component of Social awareness deals with interpersonal communication that includes both verbal and non-verbal interactions while dealing with crisis management scenarios. It governs the functioning of the OR and critical care set up, where a multidisciplinary workforce contributes significantly to the positive outcome of patients. Relationship management is the fourth integral component, which defines the role of an individual in an organisation. This refers to the ability of an individual to understand the views of other colleagues and workers.
High EI in a personality has been associated with good leadership qualities with a tendency to motivate the team members and create an empowering atmosphere in critical situations leading to improved team dynamics, decision making, clinical skills, mutual trust, information sharing, healthy risk-taking abilities, patient satisfaction and problem-solving abilities. In addition, EI has been demonstrated to have a strong positive correlation with the academic performance of the undergraduate and PG students. Therefore, incorporation of EI modules in the teaching curriculum has been recommended and initiated in most of the residency programmes.[26]
Assessment of NTS
Understanding the importance of human factors and NTS influencing medical performance, various assessment tools to measure NTS have been developed. Gaba and colleagues[27] evaluated the rating of performance and inter-rater reliability of rating made by multiple independent observers viewing videotapes of simulated crisis management. This study revealed that performance assessment tools might be useful for educational research for tracking resident's progress. The first NTS framework specifically designed for anaesthesiologists was introduced thereafter. This study discussed various emerging issues in relation to the introduction of behaviourally anchored rating scale (BARS) for assessing NTS approach in the anaesthesiologist.[28] A new set of assessment scales for four markers of NTS in simulated medical emergencies was introduced by Weinger and colleagues in 2017.[29] This study identified performance gaps, stressing the importance of medical education activities. Watkins and colleagues directly compared ANTS with BARS and suggested that BARS tool can be an alternative to the ANTS scale for the formative assessment of NTS in anaesthesiologists.[30]
A recent systematic review comparing assessment tools for NTS and their measurement properties, including 14 studies with 7 tools, concluded that the ANTS tool was most commonly assessed and appeared to be an acceptable assessment tool. However, more research was needed to determine its properties, validity and reliability across various clinical contexts.[31] Moll-Khosrawi et al.[32] conducted a study to develop and validate the anaesthesiology students NTS (AS-NTS) as a feasible rating tool to assess students’ NTS in emergency and anaesthesiology education. They concluded that AS-NTS provided a structured approach to the assessment of NTS with accurate feedback.
The impact of simulation-based training in NTS for anaesthesiologists during residency has been found to be positive. Moreover, simulation also helps in the recognition of error, enriching the value of self-confidence and the crucial role of behavioural skills.[33] High fidelity simulations can help not only in teaching of both TS as well as NTS, but also incorporate the techniques essential to ACRM. However, there may be barriers to these simulations and thus alternatives like in situ simulation, classroom simulation, tele-simulation, screen-based simulation and game-based simulations may be used.
SUMMARY
Lack of NTS has consistently been responsible for a majority of errors and adverse patient outcomes in the peri-operative period. Growing concerns about patient safety have resulted in an increased interest in the concept of NTS among anaesthesiologists. Inclusion and evaluation of NTS along with TS in the anaesthesia curriculum will go hand in hand in improving patient outcomes and should remain a continuous process. Future research should examine the measurement properties of the tools available to assess NTS in various clinical settings.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
REFERENCES
- 1.Howard SK, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anaesthesia crisis resource management training: Teaching anaesthesiologists to handle critical incidents. Aviat Space Environ Med. 1993;63:763–70. [PubMed] [Google Scholar]
- 2.Fletcher GC, McGeorge P, Flin RH, Glavin RJ, Maran NJ. The role of non-technical skills in anaesthesia: A review of current literature. Br J Anaesth. 2002;88:418–29. doi: 10.1093/bja/88.3.418. [DOI] [PubMed] [Google Scholar]
- 3.Gjeraa K, Jepsen RM, Rewers M, Ostergaard D, Dieckmann P. Exploring the relationship between anaesthesiologists’ non-technical and technical skills. Acta Anaesthesiol Scand. 2016;60:36–47. doi: 10.1111/aas.12598. [DOI] [PubMed] [Google Scholar]
- 4.Jones CPL, Fawker-Corbett J, Groom P, Morton B, Lister C, Mercer SJ. Human factors in preventing complications in anaesthesia: A systematic review. Anaesthesia. 2018;73:12–24. doi: 10.1111/anae.14136. [DOI] [PubMed] [Google Scholar]
- 5.Flin R, Fioratou E, Frerk C, Trotter C, Cook TM. Human factors in the development of complications of airway management: Preliminary evaluation of an interview tool. Anaesthesia. 2013;68:817–25. doi: 10.1111/anae.12253. [DOI] [PubMed] [Google Scholar]
- 6.Mehdiratta L, Singh Bajwa SJ. Vision, mission and formulation of newer approaches and goals at IJA: The scientific journey will continue. Indian J Anaesth. 2020;64:1–3. doi: 10.4103/ija.IJA_938_19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Wacker J, Kolbe M. Leadership and teamwork in anaesthesia- Making use of human factors to improve clinical performance. Trends Anaesth Crit Care. 2014;4:200–5. [Google Scholar]
- 8.Gaba DM, Howard SK, Small SD. Situation awareness in anaesthesiology. Hum Factors. 1995;37:20–31. doi: 10.1518/001872095779049435. [DOI] [PubMed] [Google Scholar]
- 9.Schulz CM, Endsley MR, Kochs EF, Gelb AW, Wagner KJ. Situation awareness in anaesthesia: Concept and research. Anesthesiology. 2013;118:729–42. doi: 10.1097/ALN.0b013e318280a40f. [DOI] [PubMed] [Google Scholar]
- 10.Schulz CM, Krautheim V, Hackemann A, Kreuzer M, Kochs EF, Wagner KJ. Situation awareness errors in anaesthesia and critical care in 200 cases of a critical incident reporting system. BMC Anesthesiol. 2016;16:4. doi: 10.1186/s12871-016-0172-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Greig PR, Higham HE, Darbyshire JL, Vincent C. Go/no-go decision in anaesthesia: Wide variation in risk tolerance amongst anaesthetists. Br J Anaesth. 2017;118:740–6. doi: 10.1093/bja/aew444. [DOI] [PubMed] [Google Scholar]
- 12.Anderson JG, Kathleen Abrahamson K. Your health care may kill you: Medical errors. Stud Health Technol Inform. 2017;234:13–7. [PubMed] [Google Scholar]
- 13.Chew ST, Tan T, Tan SS, Ip-Yam PC. A survey of patients’ knowledge of anaesthesia and perioperative care. Singapore Med J. 1998;39:399–402. [PubMed] [Google Scholar]
- 14.Kirschbaum C, Hellhammer DH. Salivary cortisol in psychoneuroendocrine research: Recent developments and applications. Psychoneuroendocrinology. 1994;19:313–33. doi: 10.1016/0306-4530(94)90013-2. [DOI] [PubMed] [Google Scholar]
- 15.Peng F, Peng T, Yang Q, Liu M, Chen G, Wang M. Preoperative communication with anaesthetists via anaesthesia service platform (ASP) helps alleviate patients’ preoperative anxiety. Sci Rep. 2020;10:18708. doi: 10.1038/s41598-020-74697-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ji L, Zhang X, Fan H, Han M, Yang H, Tang L, et al. drawMD APP-aided preoperative anaesthesia education reduce parents anxiety and improve satisfaction. Patient Educ Couns. 2016;99:265–70. doi: 10.1016/j.pec.2015.08.027. [DOI] [PubMed] [Google Scholar]
- 17.Schwendimann R, Blatter C, Dhaini S, Simon M, Ausserhofer D. The occurrence, types, consequences and preventability of in-hospital adverse events –A scoping review. BMC Health Serv Res. 2018;18:521. doi: 10.1186/s12913-018-3335-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Katz D, Blasius K, Isaak R, Lipps J, Kushelev M, Goldberg A, et al. Exposure to incivility hinders clinical performance in a simulated operative crisis. BMJ. 2019;28:750–7. doi: 10.1136/bmjqs-2019-009598. [DOI] [PubMed] [Google Scholar]
- 19.Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, et al. Safe surgery saves lives study group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–9. doi: 10.1056/NEJMsa0810119. [DOI] [PubMed] [Google Scholar]
- 20.Joint Commission on Accreditation of Healthcare Organizations. Comprehensive accreditation manual for hospitals: The official handbook, management of the environment of care chapter. (Revisions appear in italics and become effective Jan. 1, 1998.). Jt Comm Perspect. 1997;17:EC7–9. [PubMed] [Google Scholar]
- 21.Myatra SN, Patwa A, Divatia JV. Critical language during an airway emergency: Time to rethink terminology? Indian J Anaesth. 2020;64:275–9. doi: 10.4103/ija.IJA_214_20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Frich JC, Brewster AL, Cherlin EJ, Bradley EH. Leadership development programs for physicians: A systematic review. J Gen Intern Med. 2015;30:656–74. doi: 10.1007/s11606-014-3141-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Taylor CA, Taylor JC, Stoller JK. Exploring leadership competencies in established and aspiring physician leaders: An interview-based study. J Gen Intern Med. 2008;23:748–54. doi: 10.1007/s11606-008-0565-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.O’Connor PJ, Hill A, Kaya M, Martin B. The measurement of emotional intelligence: A critical review of the literature and recommendations for researchers and practitioners. Front Psychol. 2019;10:1116. doi: 10.3389/fpsyg.2019.01116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Sax HC, Gewertz BL. Understanding emotional intelligence and its role in leadership. In: Kribee MR, Chen H, editors. Leadership in Surgery. Switzerland: Springer; 2015. pp. 67–78. [Google Scholar]
- 26.Nicolaides M, Cardillo L, Theodoul I, Hanrahan J, Tsoulf M, Athanasio T, et al. Developing a novel framework for non-technical skills learning strategies for undergraduates: A systematic review. Ann Med Surg. 2018;36:29–40. doi: 10.1016/j.amsu.2018.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Gaba DM, Howard SK, Flanagan B, Smith BE, Fish KJ, Botney R. Assessment of clinical performance during simulated crises using both technical and behavioral ratings. Anesthesiology. 1998;89:8–18. doi: 10.1097/00000542-199807000-00005. [DOI] [PubMed] [Google Scholar]
- 28.Flin R, Patey R, Glavin R, Maran N. Anaesthetists’ non-technical skills. Br J Anaesth. 2010;105:38–44. doi: 10.1093/bja/aeq134. [DOI] [PubMed] [Google Scholar]
- 29.Weinger MB, Banerjee A, Burden AR, McIvor WR, Boulet J, Cooper JB, et al. Simulation-based assessment of the management of critical events by board-certified anaesthesiologists. Anesthesiology. 2017;127:475–89. doi: 10.1097/ALN.0000000000001739. [DOI] [PubMed] [Google Scholar]
- 30.Watkins SC, Roberts DA, Boulet JR, McEvoy MD, Weinger MB. Evaluation of a simpler tool to assess nontechnical skills during simulated critical events. Simul Healthc. 2017;12:69–75. doi: 10.1097/SIH.0000000000000199. [DOI] [PubMed] [Google Scholar]
- 31.Boet S, Larrigan S, Martin L, Liu H, Sullivan KJ, Etherington N. Measuring non-technical skills of anaesthesiologists in the operating room: A systematic review of assessment tools and their measurement properties. Br J Anaesth. 2018;121:1218–26. doi: 10.1016/j.bja.2018.07.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Moll-Khosrawi P, Kamphausen A, Hampe W, Schulte-Uentrop L, Zimmermann S, Kubitz JC. Anaesthesiology students’ Non-Technical skills: Development and evaluation of a behavioural marker system for students (AS-NTS) BMC Med Educ. 2019;19:205. doi: 10.1186/s12909-019-1609-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Maio Matos F, Martins MR, Martins I. Non-technical skills progression during anaesthesiology residency in Portugal: The impact of a national pedagogical plan. Med Educ Online. 2020;25:1800980. doi: 10.1080/10872981.2020.1800980. [DOI] [PMC free article] [PubMed] [Google Scholar]