Training of airway management for anesthesia teams – Measurement of transfer into daily work routine by questionnaire

Abstract

Theory

Problems in airway management are rare in anesthesia but when they occur, they have serious consequences for the patient. For this reason, training is recommended for professionals involved in anesthetic care. Here we investigated, if a newly developed technical/ non-technical hybrid airway training would be relevant for daily practice in a tertiary referral hospital.

Hypotheses

We hypothesized that: (a) both parts of the validated questionnaires meet the quality criteria for the application in anesthesia teams, (b) even though the team regularly deals with airway management, airway management training is relevant to all professions and (c) contents of the developed training can be integrated into the behaviour of the teams.

Method

In this observational study, 104 professionals took part in a one-day technical/non-technical hybrid airway training programme. Participants received a questionnaire six months after training, based on selected scales of the validated tools; “Training Evaluation Inventory” and “Transfer Climate Questionnaire”.

RESULTS

The scales of “perceived usefulness”, “task cues” and “positive reinforcement” showed good internal consistency and all were rated higher than 3.9 on a 5-point Likert scale (1=complete rejection; 5=fullest approval). The scale “negative reinforcement and punishment” showed satisfactory internal consistency for physicians (rated 2.75 ± 0.8). By removing an item in each case, the scales “attitude towards training” (rated 4.93 ± 0.2) and “extinction” (rated 3.02 ± 0.8) showed satisfactory internal consistency for nurses and anesthetic technicians. “Social Cues” did not meet qualitative criteria. There was no difference in the assessment by the professional groups.

CONCLUSIONS

The presented training course was perceived as useful by both professional groups equally, which supported the interprofessional concept. The content was positively reinforced in practice six months after training and is relevant for professionals who are regularly confronted with the topic “airway management”. Scales which meet qualitative criteria for only one profession and the scale “social cues” should be reconsidered in the context of an interprofessional team.

Introduction

Although airway management is the core competence of anesthesiologists it is often a challenge. The indications vary from patients receiving general anesthesia to allow routine surgery, to critical emergency situations with patients in acute respiratory failure. A retrospective study showed that in a six year period up to 2.5% of anaesthesia-related deaths resulted from intubation related problems. However, the overall anaesthesia-related mortality was clearly attributed only to 241 (2.2/ 1.000.000) patients ¹ . Therefore, the risk assessment of airway management based on anesthesia-related deaths seems to be difficult. For example, a prolonged hypoxic period during respiratory arrest does not necessarily lead to an increased mortality but has a negative impact on the outcome of patients. From an industrial psychology perspective, the professionals of the “anaesthesiology” work environment fulfil the conditions of a High-Responsibility-Team (HRT) ² . In order to deal with critical situations in their work environment, competence in several technical and non-technical skills is required ³ . Improvements in anesthesiologists’ technical competencies as a result of simulation based training have previously been shown ⁴ and participation in such training is requested by various European anesthesia societies ⁵ .

Concept development, preparation and maintenance of material further increase the cost of training. In Germany, these indirect expenses can amount to 60% of the total training costs ⁶ . These conditions require an evaluation of training effectiveness and efficiency far beyond whether or not the level of training was “interesting”. The best-known evaluation criteria are the four outcome levels of Kirkpatrick, which refer to: reactions, learning, behavioural change, and organizational results ⁷ . Ritzmann et al. developed an instrument for training evaluation with what they termed outcome dimensions (e.g. perceived usefulness and attitude towards training) in relation to the levels of evaluation from Kirkpatrick. The Training Evaluation Inventory (TEI) can easily be utilised by questionnaire and showed a relation to training outcomes in terms of knowledge of training content and teamwork skills ⁸ . Furthermore, Thayer et al. dealt with the question of whether or not the contents of training has been integrated in the daily routine. They proposed a model to measure the aspects of transfer that occur during the training and a favourable climate for transfer in the post-training setting ⁹ . Both validated systems were implemented into the questionnaire for our study.

The aim of this study was to evaluate the relevance and transfer of a self-developed airway training (hybrid-including technical skills and non-technical skills) for employees of an anesthesia department in a German university hospital. The course was designed to reflect the department's airway management standards.

Our hypotheses are; (a) both parts of the validated questionnaires meet the qualitative criteria for application in anesthesia teams, (b) even though the teams regularly deal with airway management, the suggested airway management training is applicable to all relevant professions and (c) the course contents and material can be integrated into team behaviour.

Materials and methods

Study Design and Population

The study was an observational study of participants of a hybrid (technical/ non-technical) airway training course that is part of the HAINS (Hannoverscher Anaesthesie-, Intensiv- und Notfall-Simulator) Safety education program. This training program focuses on the multi-disciplinary management of in-hospital emergencies. After a six month period, in which the learned contents could be applied, the participants received a link to our questionnaire via email. The period of six months was chosen, because contents, which were still being used after this period, demonstrated long-term information retention ^10–12. This study was reviewed and approved by the Ethics Committee of the Medical School Hannover (7511-2017).

Setting and Population

Inclusion criteria required all participants to be active in the field of anesthesiology at the time of the study. Participation was voluntary and could be withdrawn at any time without obligation. The survey was conducted using the online survey platform SurveyMonkey (SurveyMonkey, San Mateo, USA).

Seven days after the initial questionnaire invitation, participants received a reminder to complete the survey by email. The questionnaire could be completed using an internet browser on any available computer (workplace or at home). The university hospital had 31.963 anesthetic patients for treatment by 90 nurses and 141 physicians in 2018. All surgical disciplines are present in the hospital, especially those which anticipate difficulties in airway management (e.g. otorhinolaryngology, oral- and maxillofacial surgery). A standard operation procedure as recommended by the German national guidelines for airway management is implemented. All equipment required for the management of the difficult airway is assembled and ready in a trolley during every operation ¹³ . In Germany, an anesthesiologist and a nurse or anesthesia technician perform anesthesia as a team. If the anesthesiologist is not yet board certified, a consultant is always within earshot.

Development of the hybrid airway training course

The course was designed by a local group of experts for the target group, i.e. anesthesiologists, nurses, and anesthesia technicians, and based on daily work routine experience. The description “hybrid” was chosen, because the development follows the idea of integrating both technical and non-technical skills. Both of which are necessary to manage an emergent airway situation ¹⁴ . The training was “face to face” and took place in a center dedicated to medical simulation. The course content, teaching methods and session duration are shown in Table 1. The ratio of faculty to participants was 1:4, with a total participant size of 12 (equal physicians and nurses).

Table 1.

Content and teaching methods of the HAINS Airway Course. The content was developed from a local expert group based on their daily work experience (HAINS: Hannoverscher Anaesthesie-, Intensiv- und Notfall-Simulator).

Time Scheduling	Method	Content	Learning targets
10 minutes	Classroom talk	Welcome, learning targets	The participant knows the courses learning targets.
45 minutes	Lecture	Airway management algorithm and standard operating procedure	The participant knows the airway management guidelines.
135 minutes	Workshop Three groups in parallel, rotation after 45 minutes	A) Video laryngoscopy B) Cricothyrotomy C) Awake fibreoptic intubation	The participant knows the airway devices. The participants are able to perform the airway manoeuvres at a skill trainer.
135 minutes	Workshop Three groups in parallel, rotation after 45 minutes	D) Case discussion: “Just a routine operation” E) Laryngeal mask and supraglottic airway devices F) Endotracheal tube exchange and extubation	The participant reflects the non-technical skills in airway management. The participant knows the airway devices. The participant is able to perform the airway manoeuvres at a skill trainer.
45 minutes	Simulation and debriefing	Scenario: Expected difficult airway in the operation theatre	The participant reflects the non-technical skills in airway management. The participant is able to handle a difficult airway in a full-scale simulation.
45 minutes	Simulation and debriefing	Scenario: Exchange of a double lumen tube in a patient with a difficult airway	The participant reflects the non-technical skills in airway management. The participant is able to handle a difficult airway in a full-scale simulation.
45 minutes	Simulation and debriefing	Scenario: Unexpected difficult airway during a resuscitation situation in the emergency department	The participant reflects the non-technical skills in airway management. The participant is able to handle an unexpected difficult airway in a full-scale simulation.
20 minutes	Classroom talk	Feedback, course evaluation

During the workshops, all participants were trained in the same technical procedures. Thereafter the participants performed three simulation scenarios (duration 15 minutes), with each scenario followed by a debriefing session (30 minutes) led by instructors of both professional groups. In the scenario, the participants were exclusively deployed in areas of responsibility of their professional group. The training area was a simulated environment of an ICU (Intensive Care Unit) or anesthesia workplace with a patient simulator (SimMan 3G, Laerdal Medical, Norway), an airway trolley ¹³ and an anesthesia machine as used in the hospital (Primus IE, Dräger Medical, Germany).

Questionnaire

Selected scales of two valid questionnaires were applied to evaluate the uptake of the course content relevance to clinical setting. We focused only on the most relevant scales for analyzing the hypotheses in order to keep the questionnaire short and to obtain a realistic number of answerable questions during the routine work. The focus of the study was: usefulness, transfer of knowledge, and trained skills.

The questionnaires registered professional group and gender of the participants. From the Training Evaluation Inventory (TEI) two scales (five-point Likert Scale from 1 to 5; 1= complete rejection; 5= fullest approval) were taken. An example item is: “I can apply the content of this simulation training to my job”. This instrument was chosen as it was validated by the German language and German cultural sphere ⁸ . The two scales of interest were “perceived usefulness” and “attitude towards training” and covered training outcomes on the first level of Kirkpatrick´s four levels of training evaluation. Internal consistency of the scales are perceived usefulness (α =.89) and attitude towards training (α =.81) (8).

From the Transfer Climate Questionnaire, already used in other studies evaluating crisis resource management interventions in High-Responsibility-Teams (HRT) ¹⁵ , five scales (five-point Likert Scale from 1 to 5; 1= complete rejection; 5= fullest approval) were taken. The scales of interest covered “cues”, “reinforcements” and “extinction possibilities” and represented Kirkpatrick's level three of evaluation (behaviour). Internal consistency of the subscales inherited from the literature were social cues (α = 0.91; 2 items), task cues (α = 0.26; 2 items), positive reinforcement (α = 0.81; 3 items), negative reinforcement (α = 0.59; 3 items) and extinction (α = 0.78; 2 items). An example item is: “Jobs are designed so that I can do the job the way I was trained.” All 19 items are completely shown in appendix 1.

Statistical Analysis

Demographic and survey data were analyzed in a descriptive manner. The internal consistencies of the scales were determined by Cronbach's alpha. The normal distribution of the values were reviewed using the Kolmogorov-Smirnov test. In order to test differences in training evaluation, we conducted a one-way ANOVA.

In order to test hypothesis (c), a t-test for independent samples was conducted. We assumed p<0.05 as being statistically significant. All calculations were made using SPSS Statistics 25 (IBM Corporation, USA).

Results

In nine courses, 50 physicians and 54 nurses and anesthesia technicians participated. Overall, the survey was completed by 58 participants (return rate 55.8%). Table 2 shows the demographic data.

Table 2.

Demographic data of the survey participants. In total 58 participants completed the questionnaire (104 trained).

Occupational group	male	female	no answer
All	29 (50%)	28 (48,3%)	1 (1,7%)
Physicians
Consultant	5 (35,7%)	9 (64,3%)	0
Physician in training	8 (50%)	8 (50%)	0
Nurses
Nurse	4 (57,1%)	3 (42,9%)	0
Anaesthesia nurse	6 (42,9%)	7 (50%)	1 (7,1%)
Anaesthesia technician	2 (71,4%)	5 (28,6%)	0

Internal consistency of the scales

In order to test hypothesis (a) Cronbach´s alpha was analysed (Table 3). The scales of “perceived usefulness”, “task cues” and “positive reinforcement showed good internal consistency for both professional groups. By removing an item in each case, the scales “attitude towards training” and “extinction” showed satisfactory internal consistency for nurses and anesthesia technicians. The scale “negative reinforcement and punishment” showed satisfactory internal consistency for physicians after removing the item “colleagues ridicule the use of methods taught in training”.

Table 3.

Internal consistency of the scales with Cronbach`s alpha. Three scales showed good internal consistency, two could increase by removing an item for a professional group. Bold printed values show good internal.

Item	Cronbach`s alpha	Cronbach`s alpha (physicians)	Cronbach`s alpha (nurses)	ANOVA
Perceived usefulness	0.825	0.778	0.869	F(1,57)=0.089, p=0.767
Attitude towards training	0.391	0.466	0.347	F(1,57)=0.719, p=0.400
Removed: “I will apply what I learned to my day-to-day work."	0.547	0.494	0.658
Social cues	0.404	0.468	0.408	F(1,57)=0.089, p=0.766
Task cues	0.785	0.810	0.754	F(1,57)=2.267, p=0.138
Positive reinforcement	0.776	0.840	0.713	F(1,57)=0.638, p=0.428
Negative reinforcement and punishment	0.378	0.502	0.272	F(1,57)=0.242, p=0.625
Removed: „Colleagues ridicule the use of methods taught in training…“	0.542	0.601	0.552
Extinction	0.588	0.519	0.690	F(1,57)=0.488, p=0.488

Evaluation by profession

To evaluate the attitude and reaction towards the training, the arithmetic mean was calculated within each scale. The results were: “perceived usefulness” 4.81 ± 0.42, “attitude towards training” 4.93 ± 0.22, “social cues” 4.00 ± 0.66, “task cues” 3.75 ± 0.72, “positive reinforcement” 3.9 ± 0.68, “negative reinforcement and punishment” 2.7 ± 0.86. and “extinction” 3.01 ± 0.83. Values of the professional groups are shown in Figure 1. There were no significant differences in the evaluation of the training course between the individual professional groups.

Figure 1.

The evaluation the participants shows strong approval for the first four scales. There is no difference between the professional groups.

Discussion

Questionnaire Application

The aim of our study was to evaluate a hybrid technical- and non-technical airway management training (reflecting the standard of the department in airway management) and investigate the transfer to clinical routine. In order to test hypothesis (a), the scales require quality consideration. The scales “perceived usefulness”, “task cues” and “positive reinforcement” showed good internal consistency. These results support applicability to the organizational reality of nurses and physicians in an anesthesia department. Surprisingly, the scale “attitude towards training” does not show satisfactory values. Perhaps the individual experience of the provider in regard to “cannot ventilate – cannot intubate” situations influences the rating of the relevance. Positive experience with other approaches to airway management than those included in the course might be another reason. Regarding nursing staff, satisfactory internal consistency could be achieved by removing the item “I will apply what I have learned”. During the course, all participants had to carry out all procedures. This also means that nurses must perform a surgical cricothyrotomy on a simulator, although they will probably never perform it in a real emergency situation. The scales “social cues” did not meet quality recommendation - the scales “negative reinforcement” and “extinction” only applied to one profession. According to our interpretation, these values are the result of interprofessional collaboration in the anaesthesiology environment. The term ‘supervisor’ can be interpreted as the relationship one has to the organizational superior or the medical superior (physician to nurse). Even the physicians in further education are embedded in a system of hierarchical levels of different anesthesia specialists in case of a difficult airway (anesthesia specialist, senior physician, head of department). The method “Transfer Climate Questionnaire” Model was initially developed within the context of flight engineer training ⁹ and does not take this complex work structure into account. We recommend a development of these scales with clear terms for the interprofessional context. The rating of the perceived usefulness of the training was very high (mean 4.83), even though the content is daily business for the participants. This result is important, as utility reactions are more strongly related to learning and transfer and seem to be good a predictor of skill performance at work ^{16, 17}.

It should be noted, however, that anesthesia teams often perform airway management using endotracheal intubation techniques. Performing an emergency cricothyrotomy is very rare and stands as a last resort in airway management algorithms ¹⁸ . Skill training and discussion about different airway management techniques facilitate the formation of an internal standard operating procedure and make implementation easier ¹⁹ .

Relevance of the course

It cannot be assumed, that professionals deal with the subject matter as content in further training, merely because airway management is part of their daily business (hypothesis (b)). It is well known from resuscitation training research that acquired skills degrade after about seven months ²⁰ . For this reason, the Helsinki declaration of patient safety also requires airway training on a regular basis ⁵ . The scale “perceived usefulness” is rated high. The use of hybrid courses is established in the field of airway training ²¹ . Several types of skills, broad knowledge and lifelong training and practice is required ¹⁴ . This evaluation of the course indicates that the users are aware of this fact and consider airway management training to be useful.

Transfer to behaviour of the team

A perequisite for a transfer into clinical routine is the availability of the equipment and organizational conditions (e.g. staff and time) that are present in our course. The rating was good (mean 3.90), so we consider this aspect to be fulfilled.

In the sense of constructive alignment, the curriculum theory and practical techniques should also be applicable to realities, which will be faced. Otherwise, the investment into the course is a waste of time and money. Therefore, an internally developed course cannot allow participant employees to transfer what has been learnt to their respective working environments without first being tested. The rating of positive reinforcement was also good (mean 3.98). The scales evaluate the application of the course content in terms of collaboration with colleagues. In the healthcare system, there is a known problem with the implementation of standards in the clinical setting ²² . Standard guided treatment can save resources, especially in the event of an incident. In the “anaesthetist's non-technical skills” inventory, these characteristics have been included as a success factor in task management ²³ . The standardized approach to the subject of airway management serves to create a shared mental model. A shared mental model which describes a common understanding in terms of material to be used and team interaction ²⁴ . Common or shared mental models improve team performance in anesthesia ²⁵ . According to our interpretation of these results, the course manages to confirm the theoretical standards in the working reality, which is evaluated as positive by the colleagues.

Even if the scales „extinction” and “negative reinforcement and punishment” show a satisfactory internal consistency for only one professional group, they have been assessed as the lowest level. This indicates that employees are not admonished or actively advised on how to follow the practical steps of airway management. Several backgrounds are conceivable as reasons for these circumstances. First, as mentioned above, respiratory problems occur very rarely ¹ . Therefore employees are not often confronted with a critical situation. If deviations rarely affect the patient´s condition, colleagues are less likely to be alerted to compliance. There are many reasons coworkers choose not to make any objections or suggestions to colleagues for instance due to the corporate culture in hospitals. Grande et al. have investigated this topic as the so-called “speaking up” (intentionally expressing work-related ideas, information and opinions) and found e.g. fear of revenge, fear of embarrassment and steep hierarchy levels as a reason ²⁶ .

Furthermore, if difficulties are foreseeable, in Germany it is up to the treating physician to decide which method is used to secure an airway. Although we have described the advantages of a shared mental model above, there is a high acceptance of individualized medicine. This leads to the fact that for organizational reasons and time pressure, guidelines and standards are not observed. In the anesthesia critical incident reporting system, cases with deviation in airway management are described, which had far-reaching consequences (emergency cricothyrotomy) for the patient ²⁷ . Individuals have a large impact on successful introduction of standards ²⁸ . In addition to training, the implementation of scientific findings and standards require not only training but also a package of individual stage of change, leadership and feedback ²⁹ . In regard to the transfer of course content, it would be interesting to see whether both scales would be rated higher depending the proportion of trained employees and/ or time of implementation.

There is no significant difference in evaluation between the professional groups. This is not a matter of course, as the responsibility for treatment and the legal consequences of errors lies almost exclusively with the physician. Here seems to be a high identification of the nurses with the success of treatment. The scenarios should show that the shared mental models help and only a team can cope with a challenging airway situation. We rate the equally high rating by nurses and physicians as a success of this approach.

Limitations

This study has the limitation of a monocentric design. Changes e.g. in the safety culture of individual departments could have an impact on the investigated topic. According to the specification of the university and the ethics committee, participation in the course and study was voluntary. A positive selection of participants in regard to the survey was possible.

To avoid the experience of blame (content of the course is actually a prerequisite for the job), we did not ask any questions about the department´s standards prior to the course. Therefore, we could not describe what the participants learned newly in theoretical and practical topics.

Through standby duty systems and shift-based systems, the professionals work as a team again and again in the most diverse combination. The individuals can cooperate with both trained and untrained colleagues, so the evaluation of the participants can vary between general impression and individual experiences. The perceived uptake of knowledge has not yet proven to improve patient care. Due to the rarity of airway management difficulties, large registry studies are likely necessary to show an effect. The effort of conducting such a study is justified because of the severity of adverse outcomes due to poor airway management.

Conclusions

The scales, measuring training outcome for anesthesia teams are partially applicable to measure the transfer into the daily routine. Scales to describe “negative reinforcement and punishment” should be reconsidered. Even if the employees of an anesthesia department have contact with the topic of the difficult airway on a regular basis and Standard Operating Procedures are available, the practical training is considered very important. The participants are positively supported in everyday clinical practice when they apply skills learned.

The training is rated equally important by all professional groups and supports the concept of interprofessional training. Large registry studies are required to investigate the impact of Airway management Training.

Abbreviations

ANOVA:

analysis of variance

HRT:

High-Responsibility-Team

TEI:

Training Evaluation Inventory

Appendix

Appendix 1.

The questionnaire was compiled from the “Training Evaluation Inventory” and "Climate for Transfer Model” and filled by the participants six month after the course was announced.

[APP]	German items	English items
scale 1	Angenommener Nutzen des Training	Perceived usefulness
1	Ich finde das Simulationstraining nützlich für meinen Beruf.	I find the simulation training useful for my job.
2	Zeit in dieses Simulationstraining zu investieren war sinnvoll.	Investing time in this simulation training was useful.
3	Ich kann die Inhalte des Simulationstrainings in meinem Beruf anwenden.	I can apply the content of this simulation training in my job.
4	Ich ziehe persönlichen Nutzen aus diesem Simulationstraining	I derive personal use from this simulation training.
scale 2	Einstellung zum Training	Attitude towards training
1	Ich werde das Gelernte im beruflichen Alltag anwenden.	I will apply what I learned to my day-to-day work.
2	Ich finde es gut, dass die angesprochenen Themen vermittelt, bzw. besprochen wurden.	I like that the themes were imparted and/or discussed
3	Ich würde das Simulationstraining meinen Kollegen empfehlen.	I would recommend this simulation training to my colleagues.
scale 3	Soziale Wahrnehmung	Social Cues
1	Meine Kolleg*innen können mir zeigen, wie ich die Inhalte des Trainings in der Arbeit anwenden kann.	My colleagues help me applying the concepts learned in the training at work.
2	Meine Vorgesetzten geben mir die Möglichkeit, die Inhalte des Trainings in der täglichen Arbeit anzuwenden.	Supervisors give me the chance to try out the training in the job immediately.
scale 4	Arbeitsrealität	Task Cues
1	In der Arbeit habe die Möglichkeiten mich so zu verhalten, wie ich es im Simulationszentrum gelernt habe.	Jobs are designed so that I can do the job the way we were trained.
2	Es sind ausreichen Ressourcen (Personen o.Equipment o. Zeit) vorhanden, um die Arbeit so zu erledigen, wie es in dem Simulationstraining vermittelt wurde.	The equipment (staff, time, material) allows me to use the skills gained in training.
scale 5	Positive Verstärkung	Positive Reinforcement
1	Wenn ich die Inhalte des Simulationstrainings anwende, wird die Arbeit leichter.	When I use the content of training, jobs are easier.
2	Meine Vorgesetzten erkennen es positiv an, wenn ich die Inhalte der Trainings korrekt umsetze.	Supervisors appreciate me when I do the job as taught in training.
3	Meine Kollegen merken es positiv an, wenn ich meine Arbeit so ausführe, wie in dem Simulationstraining gelernt.	Colleagues appreciate me when I do the Job as we were taught in training.
scale 6	Negative Verstärkung	Negative Reinforcement and Punishment
1	Kolleg*innen machen sich schon mal über die in dem Simulationstraining vermittelten Inhalten lustig.	Colleagues ridicule the use of methods taught in training.
2	Wenn ich die Inhalte des Simulationstrainings nicht anwende, werde ich ermahnt.	When I fail to use the skills learned in training, I can expect to be reprimanded.
3	Um meine Arbeit richtig auszuführen, muss ich mich genau an die Techniken und Verfahrensweisen halten, die ich in den Trainings erlernt habe.	To do my job correct, I have to follow the techniques and procedures learned in training.
scale 7	Extinktion	Extinction
1	Für meine Vorgesetzten ist es eher nebensächlich, ob ich die Inhalte des Simulationstrainings anwende, solange die Arbeit erledigt wird.	Supervisors do not care if I use the skills learned in training, as long as I get the job done.
2	Ich habe wenige Gelegenheiten, die Inhalte der Simulationstrainings anzuwenden, so dass ich diese nicht vollständig verinnerlichen kann.	I have so little chance to use some of the skills learned since training, that I probably could't perform them now.