Identifying positive and negative use of non-technical skills by anesthesiologists in the clinical operating room: An exploratory descriptive study

Abstract

Background

Teamwork is a critical competency in high-risk settings like the operating room (OR). While conventional approaches focus on describing and learning from negative performance, there may be value in learning from high-performing behaviour, particularly in specialties where serious safety events are relatively rare. This study aimed to explore both the positive and negative use of non-technical skills by anesthesia practitioners in the OR and situate them within the clinical OR context.

Methods

This study employed a prospective observational design. Following research ethics approval, a sample of surgical cases in a tertiary hospital were recorded using the OR Black Box®. Data related to surgical phase timing, non-technical skills, team factors, and environmental factors were identified by analysts according to a modified Systems Engineering Initiative for Patient Safety model. We performed descriptive statistics and qualitative description of these observations.

Results

We observed 25 surgical cases capturing 242 instances of positive non-technical skills among anesthesiologists in the operating room and 9 instances of negative demonstrations. Situational awareness was most frequently (n = 160) observed, followed by communication and teamwork skills (n = 82), and were most often demonstrated in the context of potential environmental distractions (e.g., doors opening, unnecessary interruptions). The least common category of positive non-technical skills observed was leadership (n = 3).

Conclusions

Our findings show anesthesiologists are doing a lot “right” and there may be many opportunities for learning from positive practice in the clinical setting. These findings can inform future work to better understand and standardize best practices for non-technical performance in anesthesia.

1. Introduction

Teamwork has been identified as a critical competency in healthcare, especially in the high-risk operating room (OR) environment, where patient safety relies on effective interprofessional collaboration [1]. Studies have documented the rate of teamwork “failures” in the OR at 17.4 per hour [2], with adverse outcomes often arising from the accumulation of seemingly trivial errors that could have been prevented with effective teamwork [1].

While it is important to learn from cases involving “negative” performance, systematic analysis of positive performance can reduce patient harm and strengthen safe practice even further [3]. It has also been suggested that such an approach may even be more effective for learning from experience and disseminating best practices compared to the traditional focus on safety threats and negative practices [3]. This could be particularly useful in anesthesia given that anesthesia-related serious safety events are relatively rare, making it more practical to learn from the positive application of teamwork skills in everyday practice [4].

Traditional approaches to anesthesiologists' teamwork (or “non-technical”) performance rely on quantitative scoring tools (e.g., Anaesthetists' Non-technical Skills [ANTS]), which rate skills such as communication, situation awareness, and problem-solving on a numerical scale [5]. While these ratings may provide useful summative information, they do not necessarily provide a complete picture of how and under what conditions anesthesiologists are performing well (or not) in their everyday practice. With the advent of new audio-video recording technologies for the OR [6], it is now possible to systematically study teamwork in everyday practice to learn from both positive and negative performances. The first step toward generating hypotheses for future research and intervention development is to leverage this technology and an alternative approach to assessment to describe anesthesiologists’ teamwork performance. We therefore conducted an exploratory study to describe the use non-technical skills by anesthesiologists in and situate them within the clinical OR context.

2. Methods

Research ethics approval for analyzing observational OR Black Box® (Surgical Safety Technologies, Toronto, Canada) data was obtained from the Ottawa Health Science Network Research Ethics Board (20180152-01H).

2.1. Study design, setting and participants

We conducted an exploratory descriptive study. Data were collected at a tertiary hospital in Ottawa, Ontario, Canada between June and December 2019 from one OR equipped with the OR Black Box®. At the time of data collection, only laparoscopic gynecological procedures were performed in this room. This study focused on all events involving anesthesiologists, either individually or in an interaction with other team members (e.g., nurses, surgeons). Implied consent was obtained from the study participants. There were no refusals to participate among the cases collected. The details of our consent process are described elsewhere [7].

2.2. Variables

Non-technical skills were assessed based on established domains: communication and teamwork, situational awareness, decision-making, leadership, and task management [5]. Both positive and negative demonstrations of non-technical skills were recorded. Additional contextual factors surrounding the demonstration of non-technical skills were categorized according to the Systems Engineering Initiative for Patient Safety (SEIPS) model [8]. The SEIPS model is comprised of six components related to patient safety: person, tasks, tools and technology, physical environment, organization, and external environment [9]. Inter-coder agreement for this model has been previously found to be moderate (Cohen's kappa = 0.73) [9].

2.3. Data sources/measurement

We used the OR Black Box® to prospectively collect clinical intraoperative data. Eligible cases included all laparoscopic procedures that took place in the OR where the OR Black Box® was installed. The data capture and analysis process of the OR Black Box® system has been described elsewhere [7]. Briefly, the process involved synchronized capture of video and audio, as well as patient and environmental data. Cases were recorded from the time the patient entered the OR to the time they exited and were analyzed by expert raters with healthcare and human factors backgrounds along with advanced software algorithms (intraclass correlation coefficient = 0.74). An outcome report, including both quantitative and qualitative data, was then generated for subsequent statistical and qualitative analyses.

2.4. Study size

For the purposes of this exploratory study, we used a convenience sample consisting of the first 25 eligible consecutive cases recorded.

2.5. Statistical methods

Quantitative and qualitative data were summarized descriptively. Descriptive statistics for quantitative data were calculated with Excel Version 16 (Microsoft, Redmond, United States).

3. Results

3.1. Description of cases

The 25 cases involved the following procedures: total abdominal hysterectomy (n = 8), unilateral salpingo-oophorectomy (n = 1), tubal reversal (n = 1), excision of endometriosis (n = 6), salpingectomy (n = 2), diagnostic laparoscopy (n = 1), bilateral salpingo-oophorectomy (n = 2), myomectomy (n = 2), vaginal hysterectomy (n = 1), and hysteroscopy (n = 1). The median procedure duration was 2 h and 39 min, with procedures ranging from 52 min to 4 h and 6 min. No emergency cases were observed.

4. Teamwork performance

Across 25 surgical cases, we observed 242 positive demonstrations of non-technical skills among anesthesiologists in the operating room and only 9 instances of negative practices (Table 1). Across all cases, on average, one positive demonstration was observed every 12.5 min. Most of the resilience observations involved situational awareness (n = 159 [65.7%]) or communication. (n = 71 [29.3%]). Conversely, the few instances of negative practice mostly concerned shortcomings in task management (n = 8 [88.9%]). We also observed 260 safety threats (Table 2), with door openings being the most frequently observed (n = 154 [5.2%]), followed by ambient distractions (n = 79 [30.4%]). Infectious risks were rarely observed (n = 2 [1%]).

Table 1.

Teamwork performance observations (n = 25 cases).

Teamwork behaviours observed	Optimal		Suboptimal
	Total n (%)	Median (IQR) per case	Total n (%)	Median (IQR) per case
Situational awareness	159 (65.7%)	10.5 (12)	1 (11.1%)	1 (0)
Communication and teamwork	71 (29.3%)	4 (2)	0	NA
Task management	7 (2.9%)	1 (1)	8 (88.9%)	1 (0.75)
Decision making	3 (1.2%)	1 (0)	0	NA
Leadership	2 (0.8%)	1 (0)	0	NA
Total	242		9

IQR = interquartile range.

Table 2.

Safety threat observations (n = 25 cases).

Type of safety threat	Total n (%)	Median (IQR) per case
Door openings	154 (59.2%)	8 (7)
OR environment	79 (30.4%)	8 (7.25)
Changeovers	12 (4.6%)	1 (1.5)
Primary operator	5 (1.9%)	1 (1)
Operations update	5 (1.9%)	1 (0.25)
Clinical observations	3 (1.2%)	1 (0)
Infectious risks	2 (1%)	1 (0)
Total	260	11 (18)

IQR = interquartile range.

Door Openings: when a team member enters or exits the room; OR Environment: when an ambient distraction occurs in the OR (e.g., a ringing phone or a machine alarm); Changeovers: when a team member is replaced by a colleague in the same role; Primary Operator: when the primary surgeon performing the operation changes (e.g., from a staff surgeon to a resident); Operations Update: when someone asks for an update on the progress of the operation, creating time pressure; Clinical Observations: when an aspect of the surgery is performed in a suboptimal way (e.g., inserting a trocar without visualization); Infectious Risks: when an action carries a risk of contaminating the patient or materials.

Fig. 1A and B shows the specific non-technical skill categories that anesthesiologists demonstrated in the context of other OR team and environmental factors (see Appendix 1 for data in tabular format). Situational awareness was most frequently (n = 160, 61.5%) observed, particularly in the same time frame as the door opening (n = 93, 58% of these 160) or another potential environmental distraction (e.g., machine alarms, unnecessary interruptions) (n = 48, 30% of these 160). Communication and teamwork skills were also observed to co-occurred with these potential distractions, (n = 82). The least common category of positive non-technical skills observed was leadership (n = 3). Appendix 2 presents a full case example that follows the surgical case from start to finish. It lists all demonstrations of non-technical skills along with concurrent patient safety-related factors observed.

Fig. 1.

Optimal non-technical skills in context.

5. Discussion

This exploratory study identified and characterized anesthesiologists’ non-technical skills within the context of the interprofessional OR environment. Across all included surgical cases, we observed the positive use of non-technical skills more often than negative use. Situational awareness, communication and teamwork were the most frequent skills, while decision-making and leadership were the least observed. Non-technical skills have long been emphasized as critical for patient safety in the anesthesia literature [10,11], but few studies have explored the positive demonstration of these skills in the clinical context. Although studies of quality and safety in anesthesia, as well as in healthcare more broadly, tend to focus on reporting negative practices or observations from simulated cases, our findings show that anesthesiologists are doing a lot “right” and there may be many opportunities for learning from positive practice in the real OR. Foremost, it is noteworthy how often effective non-technical skills are observed both within a single case and across cases. Even when potential distractions occurred, there were numerous demonstrations of positive non-technical skills and relatively few instances of negative non-technical skills. Identifying high performers is the first of four steps proposed by Bradley et al. [12] for improving quality in health care from a “positive deviance” approach. The next steps involve using qualitative methods to determine what enables them to achieve top performance, testing these hypotheses in a larger representative sample, and disseminating best practices in partnership with key stakeholders. Our findings provide an important foundation from which to undertake this process in anesthesiology.

Another potential avenue to explore in future research may be related to the types of non-technical skills that we observed. Anesthesiologists appeared to use situational awareness, communication and teamwork substantially more than other skills. This may suggest that either these skills are the most “valuable” when responding to a potential distraction or other safety threat and/or that other skills may not be as “necessary” during routine cases. It may be important to explore whether this observed discrepancy persists across additional cases and types of surgery, including emergency cases. In turn, positive examples of situation awareness, communication and teamwork may be used to establish and teach best practices. There may also be an opportunity to extend identified supports for these skills to optimize the practice of other non-technical skills.

6. Limitations

We acknowledge that this study is based on a small sample of cases from a single center and generalizability to other surgical specialties and centres as well as to emergency cases is unknown. It is also possible, as with any study where participants are being observed, that anesthesiologists modified their behaviour when in the OR Black Box® room (i.e., Hawthorne effect). Nevertheless, our exploratory work adds to the literature as it describes the demonstration of positive non-technical skills rather than focusing only on negative events. It can inform future efforts to improve practice in anesthesia and provide a potential model that can be extended across other settings. It should be noted that we were unable to collect data on individual anesthesiologists involved in each case due to privacy regulations. However, the cases observed in this study involved a number of anesthesiologists as there are approximately 40 anesthesiologists who rotate through each OR at our centre.

7. Conclusions

Positive non-technical skills were commonly observed among anesthesiologists in this study, particularly with regard to demonstrating situational awareness, communication and teamwork. These findings can inform future work to better understand and standardize best practices for non-technical performance in anesthesia.

Ethics approval and consent to participate

The study protocol was reviewed and approved by the Ottawa Health Science Network Research Ethics Board (OHSN-REB) (Protocol # 20180152-01H). The OHSN-REB waived the need of informed consent for this study. All methods were carried out in accordance with relevant institutional guidelines and regulations.

Author contribution statement

Joseph K. Burns: analyzed and interpreted the data; wrote the paper - original draft. Nibras Ghanmi: contributed analysis tools; analyzed and interpreted the data. Cole Etherington: conceived and designed the experiments; analyzed and interpreted the data; wrote the paper. Agnes Crnic, Edward Crosby, Fadi Mansour, Christopher L. Pysyk: conceived and designed the experiments. Sylvain Boet: conceived and designed the experiments; analyzed and interpreted the data; wrote the paper.

Funding statement

This study was funded by a research operating grant obtained from the University of Ottawa Department of Anesthesiology and Pain Medicine (No grant number assigned). Dr. Boet was supported by The Ottawa Hospital Anesthesia Alternate Funds Association and the Faculty of Medicine, University of Ottawa with a Tier 2 Clinical Research Chair. The funders did not have any role in the design of the study, the collection, analysis, and interpretation of data, or in writing the manuscript.

Data availability statement

The data that has been used is confidential.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix 1. Optimal non-technical skills in the context of safety threats

Resilience Event	Safety Threat
	Door Opening	OR Environ-ment	Change-over	Primary Operator Change	Operations Update	Clinical Observations	Infectious Risk	Grand Total
Communication & Teamwork	50	25	4		2		1	82
Decision Making	3	1					1	5
Leadership	2	1						3
Situational Awareness	93	48	8	5	3	3		160
Task Management	6	4						10
Grand Total	154	79	12	5	5	3	2	260

Appendix 2. Case example of resilience events associated with contemporaneous safety threats

Procedure: Excision of endometriosis CaseID: 045 Primary operator: Staff surgeon
Surgical phase	Start – end time of surgical phase	Start – end time of resilience event	Resilience supports		Safety threats
			Observation	SEIPS Classification	Observation	SEIPS Classification
Surgical Safety Checklist	00:00:22–00:01:23	N/A	N/A	N/A	N/A	N/A
Anesthesia Setup	00:00:00–00:07:47	N/A	N/A	N/A	N/A	N/A
Nursing Setup	00:00:00–00:17:10	00:00:00–00:01:30	Communication & teamwork (whole team): Provides briefing and clarifies objectives and goals before commencing operation.	Person – anticipatory action – establishing next steps	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		00:07:51–00:10:05	Anticipating (within anesthetic team): Keeps ahead of situation by giving fluids/drugs	Person – anticipatory action – proactive task completion	Door opened 1x Machine alarms 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds
		00:15:21–00:15:37	Communication & teamwork (whole team): Co-operates with others to achieve goals	Person – effective teamwork – team harmony	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
Placement of Monitors	00:02:06–0:05:44	N/A	N/A	N/A	N/A	N/A
Monitors until Induction	00:05:44–00:07:47	N/A	N/A	N/A	N/A	N/A
Induction	00:07:47–00:12:00	N/A	N/A	N/A	N/A	N/A
Intubation	00:12:00–00:13:06	N/A	N/A	N/A	N/A	N/A
Intubation until Prep and Drape	00:13:06–02:09:02	N/A	N/A	N/A	N/A	N/A
Positioning	00:15:57–00:24:26	N/A	N/A	N/A	N/A	N/A
Preop Counts	00:17:10–00:23:55	N/A	N/A	N/A	N/A	N/A
Prep and Drape	00:32:37: 00:46:10	N/A	N/A	N/A	N/A	N/A
Timeout	00:50:25–00:51:06	00:50:26–00:51:37	Communication & teamwork (whole team): Provides briefing and clarifies objectives and goals before commencing operation.	Person – anticipatory action – establishing next steps Organization – effective policies/procedures - timeout	Door opened 1x Machine alarms 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds
Foley	00:50:31–00:51:46	N/A	N/A	N/A	N/A	N/A
Surgical Procedure	00:52:17: 02:22:42	00:53:20–00:54:12	Situational awareness (anesthetic team): Increases frequency of monitoring in response to patient condition	Person – high-performance behaviour – evaluating circumstances	Door opened 2x Machine alarms 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds
		00:58:14–01:00:58	Situational awareness (surgeon to anesthesiologist): Optimises operating conditions before starting (e.g., moves table, lights, AV equipment)	Person – effective teamwork – shared mental model	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		01:06:53–01:07:15	Situational awareness (anesthetic team): Keeps ahead of situation by giving fluids/drugs	Person – anticipatory action – proactive task completion	Door opened 2x Machine alarms 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds
		01:07:15–01:07:49	Situational awareness: (anesthetic team): Conducts frequent scan of the environment	Person – high-performance behaviour – paying attention	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		01:08:35–01:13:02	Situational awareness: (anesthetic team): Increases frequency of monitoring in response to patient condition	Person – high-performance behaviour – evaluating circumstances	Door opened 1x Machine alarms 1x Changeover 1x: Scrub nurse	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds Organization – ineffective staff management – staff change
		01:22:19–01:22:33	Situational awareness (surgeon to anesthesiologist): Optimises operating conditions	Person – effective teamwork – shared mental model	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		01:23:31–01:28:28	Situational awareness (within anesthetic team): Increases frequency of monitoring in response to patient condition	Person – high-performance behaviour – evaluating circumstances	Machine alarms 2x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		01:33:32–01:34:07	Situational awareness (within anesthetic team): Keeps ahead of situation by giving fluids/drugs	Person – anticipatory action – proactive task completion	Door opened 1x Changeover 1x: Circulating nurse	Physical environment – suboptimal ambient conditions – distracting workflow sounds Organization – ineffective staff management – staff change
		01:37:38–01:44:31	Situational awareness (within anesthetic team): Conducts frequent scan of the environment	Person – high-performance behaviour – paying attention	Door opened 4x Machine alarms 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds
		01:52:27–01:52:47	Situational awareness (within anesthetic team): Keeps ahead of situation by giving fluids/drugs	Person – anticipatory action – proactive task completion	Operations update given (unrequested) 1x	Tasks – disruptions – unnecessary verbal interruption
		02:01:03–02:02:24	Situational awareness (within anesthetic team): Conducts frequent scan of the environment	Person – high-performance behaviour – paying attention	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		02:09:03–02:09:21	Situational awareness (within anesthetic team): Optimises operating conditions	Person – effective teamwork – shared mental model	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		02:09:25–02:11:09	Situational awareness (within anesthetic team): Conducts frequent scan of the environment	Person – high-performance behaviour – paying attention	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
		02:14:54–02:23:04	Situational awareness (within anesthetic team): Conducts frequent scan of the environment	Person – high-performance behaviour – paying attention	Door opened 2x Machine alarms 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds Physical environment – suboptimal ambient conditions – distracting workflow sounds
Postop Counts	02:14:54: 02:20:25	N/A	N/A	N/A	N/A	N/A
Emergence and Extubation	02:22:41: 02:26:24	02:26:09–02:27:56	Communication & teamwork (whole team): Debriefs relevant team members after operation, discussing what went well and problems that occurred	Person – effective teamwork – debriefing	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds
Dressings and Application of Other Devices	02:23:38: 02:24:41	N/A	N/A	N/A	N/A	N/A
Postop Debrief	02:27:10: 02:27:36	N/A	N/A	N/A	N/A	N/A
Bed Transfer	02:28:43: 02:30:28	02:30:10–02:30:14	Communication & teamwork (whole team): Co-operates with others to achieve goals	Person – effective teamwork – team harmony	Door opened 1x	Physical environment – suboptimal ambient conditions – distracting workflow sounds