Simulation-based training and assessment of non-technical skills in the Norwegian Helicopter Emergency Medical Services: a cross-sectional survey

Abstract

Background

Human error and deficient non-technical skills (NTSs) among providers of ALS in helicopter emergency medical services (HEMS) is a threat to patient and operational safety. Skills can be improved through simulation-based training and assessment.

Objective

To document the current level of simulation-based training and assessment of seven generic NTSs in crew members in the Norwegian HEMS.

Methods

A cross-sectional survey, either electronic or paper-based, of all 207 physicians, HEMS crew members (HCMs) and pilots working in the civilian Norwegian HEMS (11 bases), between 8 May and 25 July 2012.

Results

The response rate was 82% (n=193). A large proportion of each of the professional groups lacked simulation-based training and assessment of their NTSs. Compared with pilots and HCMs, physicians undergo statistically significantly less frequent simulation-based training and assessment of their NTSs. Fifty out of 82 (61%) physicians were on call for more than 72 consecutive hours on a regular basis. Of these, 79% did not have any training in coping with fatigue. In contrast, 72 out of 73 (99%) pilots and HCMs were on call for more than 3 days in a row. Of these, 54% did not have any training in coping with fatigue.

Conclusions

Our study indicates a lack of simulation-based training and assessment. Pilots and HCMs train and are assessed more frequently than physicians. All professional groups are on call for extended hours, but receive limited training in how to cope with fatigue.

Key messages.

What is already known on this subject?

• Human error and deficient non-technical skills among providers of ALS in helicopter emergency medical services (HEMS) is a threat to patient and operational safety.

• Skills can be improved through simulation-based training and assessment.

• Crew resource management is a safety management strategy, mandatory for crew members in HEMS, intended to train and assess non-technical skills.

What might this study add?

• A significant number of crew members in the Norwegian HEMS lacked simulation-based training in, and assessment of, generic non-technical skills.

• All professional groups in HEMS are on call for extended hours but receive limited training in how to cope with fatigue.

Introduction

In Norway, physician-manned air ambulance helicopters support ground ambulances in emergency missions for care and retrieval, and provide interhospital transfer of patients. The provision of ALS to critically ill and injured patients in helicopter emergency medical services (HEMS) is a complex process characterised by shifting workload and goals, ill-structured problems, uncertainty, intense time pressure, high stakes and a set of individually complex and interacting tasks of flight-operative, medical, technical, rescue and multidisciplinary character.¹ This process is prone to human error, adverse events and ultimately iatrogenic injury,^{2 3} which are to a large degree preventable.^{1 4 5} The Norwegian HEMS conduct more than 7500 urgent and interhospital air medical patient transfers annually. More than 60% of these patients are critically ill or injured (National Advisory Committee for Aeronautics (NACA) score 4–6), and more than 12% are mechanically ventilated.⁶

Major adverse events in HEMS are rare, but the overall incidence of adverse events remains unknown.² Poor interdisciplinary communication seems to be a significant factor in adverse events in air ambulance services⁷ and during trauma resuscitation.⁸ Baseline haemodynamic instability, mechanical ventilation and on-scene calls are factors associated with increased risk of life-threatening events in transit.⁹ Human error in any of these settings can be fatal.

Crew resource management (CRM) is a conglomerate of multidisciplinary, safety-management principles and training interventions designed to reduce human error by enhancing non-technical skills (NTSs).^{10 11} NTSs can be defined as ‘the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and effective task performance’.^{5 12} Seven generic categories of NTSs have been suggested: situation awareness, decision-making, communication, teamwork, leadership, managing stress and coping with fatigue.¹² Systematic training and assessment of NTSs in HEMS has received little attention in the past, although CRM training is required for all crew members. The time-pressured HEMS environment is not particularly suited for experiential training of NTSs.

Simulation-based training and assessment of NTSs, as one of several CRM training interventions, is called for and recommended.^13–16 Multiprofessional simulation allows repetitive practice in rare conditions and potentially dangerous operations in a safe environment, reinforces understanding across disciplines, and permits real teams to train based on the knowledge of challenges and deficiencies.^{15 16} Simulation-based trauma team training has shown a significant effect on learning and team performance.^15–17 Simulation is a useful tool for developing NTSs.¹⁴

The aim of this study was to document the current level of simulation-based training and assessment of a generic set of basic NTSs among crew members of the Norwegian HEMS. We hypothesised that crew members lacked simulation-based training in, and assessment of, NTSs. We also hypothesised that the extent of simulation-based training and assessment of these skills differed across the professional groups in this service.

Methods

Setting

Eleven civilian HEMS bases operate in Norway today. Work is carried out by a small team (crew). Three crew members is the main crew concept. Each individual belongs to a separate profession. All of these professionals have their own group cultures and team dynamics, with different backgrounds and expertise, and they often work together only for a short period of time. The individual with his/her professional background is the basic building block from which HEMS crews are formed.¹² In addition, team composition is continually shifting. This is why we chose to stratify our analysis by profession.

The pilot is the mission commander and has primary responsibility for flight safety and navigation; the HEMS crew member (HCM) is responsible for rescue operations and assists the physician on-scene and the pilot during flight operations; meanwhile, the physician is a certified or in-training anaesthesiologist responsible for patient treatment and care on-scene and during transportation to the hospital. Only one base operates with a nurse on board in addition to the aforementioned three-man crew. This is a local adaptation and is thus not representative of the general crew composition.

Questionnaire

Eight question categories relating to education and training in NTSs were presented as an extension of a patient safety climate questionnaire (see online supplementary appendix, section I). The present study focuses on the two question categories documenting the overall extent of simulation-based training (question I6) and assessment (question I7) on a four-point ordinal scale (0, 1–2, 3–5, >5 times per year). Both question categories contained seven questions, one for each of the aforementioned seven generic NTS categories: (1) decision-making, (2) leadership, (3) communication, (4) situation awareness, (5) teamwork, (6) managing stress and (7) coping with fatigue.¹²

The questionnaire contained information on one possible explanatory variable: the maximum number of consecutive on-call duty hours, reported on a seven-point ordinal scale.

Data collection

Between 8 May and 25 July 2012, we conducted an anonymous, cross-sectional survey among all 207 physicians, HCMs and pilots working in the civilian Norwegian HEMS. To maximise the response rate, a commentary on the upcoming study was published in the Norwegian Medical Journal.¹⁸ The survey was distributed via both e-mail, with a link to a web-based questionnaire (Questback), and an identical paper version (see online supplementary appendix) along with prepaid stamped return envelopes. After 2–4 weeks, all crew members received a follow-up phone call as a reminder and encouragement to answer.

Questionnaires returned with missing data on occupation or profession were excluded. We also excluded those with more than 50% missing values in order to maintain consistency with an upcoming psychometric analysis from other parts of the questionnaire relating to safety climate, but not within the scope of this survey (see online supplementary appendix). Respondents were excluded if they did not work in the civilian HEMS (eg, military search and rescue helicopter or aeroplane) and if they did not belong to the aforementioned target group of professionals (eg, nurses and paramedics).

Statistical analysis

Our unit of analysis is the professional groups rather than the HEMS crew as a whole. Descriptive data are presented as ratios or numbers. Spearman's correlation (r_s) was calculated to assess the inter-item association between each of the seven items in question categories I6 and I7. Frequency of simulation-based training and assessment of NTSs across all professional groups is presented as bar charts. The group of nurses was considered too small (n=6) to allow comparison of professional groups in a rigorous statistical analysis.

To assess possible differences in simulation-based training and assessment between professions, we dichotomised the items (0=no training/assessment, 1=some training/assessment) and used them as dependent variables in a series of logistic regression models, with crew type as a three-level nominal explanatory variable: physician, HCM and pilot. The last of these was used as the reference group, since the aviation industry has led the field and driven formal assessment of individual pilot’s NTSs.¹⁹ Results are presented as OR with 95% CI.

Fisher’s exact test was used to explore the association between crew members working for the health enterprise (physicians) or the flight operators (HCMs and pilots) and three dichotomised variables by using a two-by-two design: on-call duty hours (0=less than or equal to 72 h, 1=more than 72 h); simulation-based training and assessment (0=no training/assessment, 1=some training/assessment). Results are presented as ratios (%) and numbers, and a p value less than 0.05 was considered significant.

SPSS V.18.0 and the freeware R 2.12 were used for all calculations.

Ethics

This study was conducted in compliance with the ethics guidelines of the Helsinki Declaration. All participants received written information about the purpose of the study, and were told that the data would be collected anonymously and treated in confidence. The regional ethics committee of South-Eastern Norway (reference number 2010/3326) and the Norwegian Social Science Data Services reviewed and approved the study. Written informed consent was considered unnecessary, since responding to the questionnaire was voluntary.

Results

Of the 207 people working at the 11 Norwegian HEMS bases, 172 responded (150 electronically, 22 on paper via mail), of which 158 were eligible for inclusion. Accordingly, the response rate was 81.8% (figure 1). All HEMS bases were represented among the respondents. Of the included respondents, 82 (52.9%) were working for the health enterprise, and 73 (47.1%) for the flight operator. None of the HCMs had less than 5 years of prehospital experience. In contrast, 26 of the 82 physicians (31.7%) and 13 of the 32 pilots had less than 5 years of prehospital experience. Of the HCMs, 33 (78.6%) had more than 10 years of prehospital experience.

Figure 1.

Participant flow through the study illustrating inclusion/exclusion of respondents. HEMS, helicopter emergency medical service; HCM, HEMS crew member; SAR, search and rescue.

There is a strong correlation (0.68≤r_s≤0.89) between the generic NTS categories 1–6 related to the simulation-based training of NTSs (table 1, question category I6). Correlation between these six categories and the skill category ‘coping with fatigue’ was generally somewhat smaller (0.53≤r_s≤0.78). There is also a strong correlation (0.77≤r_s≤0.91) between the NTS categories 1–6 related to the assessment of NTSs (table 1, question category I7). Correlation between these six categories and the skill category ‘coping with fatigue’ was distinctly smaller (0.62≤r_s≤0.76).

Table 1.

Inter-item correlations (Spearman r, r_s) between each of the seven generic non-technical skills (NTSs) in the question categories I6 and I7

Question category	NTS category	1	2	3	4	5	6	7
I6: Simulation-based training of NTSs (144≤n≤150)	1. Decision-making	–
	2. Leadership	0.85	–
	3. Communication	0.89	0.88	–
	4. Situation awareness	0.81	0.80	0.85	–
	5. Teamwork	0.85	0.77	0.85	0.83	–
	6. Managing stress	0.77	0.71	0.71	0.74	0.68	–
	7. Coping with fatigue	0.59	0.60	0.56	0.61	0.53	0.78	–
I7: Assessment of NTSs (145≤n≤149)	1. Decision-making	–
	2. Leadership	0.91	–
	3. Communication	0.90	0.89	–
	4. Situation awareness	0.86	0.81	0.90	–
	5. Teamwork	0.88	0.87	0.83	0.82	–
	6. Managing stress	0.80	0.80	0.77	0.80	0.82	–
	7. Coping with fatigue	0.65	0.66	0.62	0.68	0.62	0.76	–

Missing values were excluded pairwise. All correlations (I6 and I7) reached statistical significance at the 0.01 level (two-tailed).

Visual inspection of the bar charts of the frequency of simulation-based training (figure 2) and assessment (figure 3) indicate that HCMs generally appear to train and undergo assessment more frequently, and physicians less frequently, than pilots.

Figure 2.

Multidisciplinary, prehospital simulation-based training of generic non-technical skills (1–7) in 2011.¹ Complete answers from each of the three professional groups in a HEMS crew (horizontal axis) across four ordinal categories of frequency within a year (box). Proportion of individuals (relative frequency, %) within each professional group on vertical axis. HEMS, helicopter emergency medical service.

Figure 3.

Assessment of seven (1–7) generic non-technical skills in 2011.¹ Complete answers from each of the three professional groups in a HEMS crew (horizontal axis) across four ordinal categories of frequency within a year (box). Proportion of individuals (relative frequency, %) within each professional group on vertical axis. HEMS, helicopter emergency medical service.

These apparent differences in simulation-based training between groups of crew members are, however, not statistically significant (table 2, question category I6). The tendency for ORs to be larger for HCMs and smaller for physicians can be seen across all skill categories, but CIs are wide.

Table 2.

OR with 95% CIs for physicians and HEMS crew members (HCMs) having undergone simulation-based training (question category I6) and assessment (question category I7) of seven (1–7) generic non-technical skills (NTSs),¹ compared with the group of pilots

		Physician			HCM
Question category	NTS category (n)	OR (95% CI)	p Value	nphys (missing)	OR (95% CI)	p Value	nHCM (missing)
I6: Simulation-based training of NTSs	1. Decision-making (n=149)	0.52 (0.22 to 1.24)	0.139	76 (6)	1.38 (0.51 to 3.72)	0.530	42 (0)
	2. Leadership (n=150)	0.74 (0.32 to 1.71)	0.486	78 (4)	2.25 (0.84 to 6.04)	0.109	41 (1)
	3. Communication (n=150)	0.68 (0.29 to 1.60)	0.379	77 (5)	1.78 (0.66 to 4.83)	0.257	42 (0)
	4. Situation awareness (n=150)	0.67 (0.29 to 1.55)	0.348	77 (5)	2.04 (0.76 to 5.48)	0.160	42 (0)
	5. Teamwork (n=149)	0.66 (0.27 to 1.58)	0.346	76 (6)	2.02 (0.69 to 5.94)	0.200	42 (0)
	6. Managing stress (n=151)	0.32 (0.14 to 0.76)	0.010	78 (4)	1.61 (0.61 to 4.24)	0.334	42 (0)
	7. Coping with fatigue (n=146)	0.46 (0.18 to 1.17)	0.103	78 (4)	1.71 (0.64 to 4.55)	0.284	40 (2)
I7: Assessment of NTSs	1. Decision-making (n=149)	0.40 (0.17 to 0.96)	0.039	77 (5)	1.49 (0.56 to 3.97)	0.428	42 (0)
	2. Leadership (n=149)	0.36 (0.15 to 0.86)	0.021	77 (5)	1.08 (0.42 to 2.83)	0.870	42 (0)
	3. Communication (n=148)	0.33 (0.14 to 0.78)	0.012	76 (6)	1.08 (0.42 to 2.83)	0.870	42 (0)
	4. Situation awareness (n=148)	0.40 (0.17 to 0.94)	0.036	77 (5)	2.12 (0.79 to 5.65)	0.136	41 (1)
	5. Teamwork (n=149)	0.37 (0.15 to 0.88)	0.025	77 (5)	1.45 (0.58 to 3.93)	0.469	42 (0)
	6. Managing stress (n=149)	0.29 (0.12 to 0.69)	0.005	77 (5)	1.24 (0.48 to 3.23)	0.655	42 (0)
	7. Coping with fatigue (n=146)	0.38 (0.15 to 0.98)	0.046	77 (5)	1.64 (0.64 to 4.34)	0.318	39 (3)

Significance at level 0.05.

Physicians are assessed significantly less frequently than pilots (table 2, question category I7), but the differences between HCMs and pilots are not statistically significant. There is, however, also a tendency here for ORs to be larger for HCMs and smaller for physicians across all skill categories.

Compared with employees working for the flight operator (pilots and HCMs), employees working for the health enterprise (physicians) undergo statistically significantly less frequent simulation-based training (table 3, question category I6) and assessment (table 3, question category I7).

Table 3.

Proportion (%) of crew members in helicopter emergency medical services (HEMS) working for the health enterprise (physicians) and for the flight operator (HEMS crew members (HCMs) and pilots) who have undergone simulation-based training (question category I6) and assessment (question category I7) of seven (1–7) generic non-technical skills (NTSs)¹

Question category	NTS category	Health enterprise employee	Flight operator employee	N (missing)	p Value (2-sided)
I6: Simulation-based training of NTSs	1. Decision-making	37/76 (48.7%)	50/73 (68.5%)	149 (6)	0.020
	2. Leadership	37/78 (47.4%)	47/72 (65.3%)	150 (5)	0.033
	3. Communication	40/77 (51.9%)	50/73 (68.5%)	150 (5)	0.046
	4. Situation awareness	37/77 (48.1%)	49/73 (67.1%)	150 (5)	0.021
	5. Teamwork	44/76 (57.9%)	55/73 (75.3%)	149 (6)	0.037
	6. Managing stress	24/78 (30.8%)	47/73 (64.3%)	151 (4)	<0.001
	7. Coping with fatigue	18/78 (23.8%)	32/68 (47.1%)	146 (9)	0.003
I7: Assessment of NTSs	1. Decision-making	29/77 (37.7%)	47/72 (65.3%)	149 (6)	0.001
	2. Leadership	27/77 (35.1%)	44/72 (61.1%)	149 (6)	0.002
	3. Communication	25/76 (32.9%)	44/72 (61.1%)	148 (7)	0.001
	4. Situation awareness	24/77 (31.2%)	45/71 (63.3%)	148 (7)	<0.001
	5. Teamwork	30/77 (38.9%)	49/72 (68.1%)	149 (6)	<0.001
	6. Managing stress	21/77 (27.3%)	43/72 (59.7%)	149 (6)	<0.001
	7. Coping with fatigue	14/77 (18.2%)	30/69 (43.5%)	146 (9)	0.001

Comparison of health enterprise employees with flight operator employees using Fisher’s exact test (two-sided). Significance at level 0.05.

All professional groups work longer hours and are exposed to significant fatigue. Of the hospital employees, 50 out of 82 (61%) were on call for more than 72 consecutive hours on a regular basis. Of these, 79% did not have any training in coping with fatigue. In contrast, 72 out of 73 (99%) pilots and HCMs were on call for more than 3 days in a row. Of these, 54% did not have any training in coping with fatigue.

Discussion

This is the first study of simulation-based training and assessment of NTSs in the Norwegian HEMS. We found considerable variation in the extent of simulation-based training and assessment of NTSs among the crew members. A significant number of crew members reported complete absence of simulation-based training and assessment.

The strength of correlations between the NTS categories was generally high. That is, the more respondents train or undergo assessment in one of the NTS categories, the more they generally train or undergo assessment in other NTS categories. The item ‘coping with fatigue’ differs from the other skill categories, which might reflect the fact that it is not an explicit skill category but rather an item that influences the others.

Lack of simulation-based training

The need for training in complex environments is often underestimated.⁴ Our data indicate that, compared with HCMs and pilots, a statistically significantly smaller proportion of HEMS physicians have undergone simulation-based NTS training. Similarly, as early as 2001, it was suggested that anaesthesiologists lacked training in NTSs for critical situations in hospitals.¹⁰ To overcome this, Gaba and colleagues created a simulation-based curriculum based on key principles from aviation CRM training.¹⁰ Differences in task environment and professional cultures may help to provide an answer to what we have revealed. It is claimed that aviation is more procedure-based than prehospital critical care, and hence it is easier to train and assess crew in its process. Airline staff also have longer traditions of recurrent training in, and evaluation of, NTSs than medical staff.¹⁰ In addition, the professional cultures differ markedly. Aviation staff have managed to change the professional culture into one that recognises human limitations and the need for NTS training,²⁰ while cultural resistance against extending CRM training into the medical domain has been reported.²¹Another obstacle to training is that simulation-based training is a time-consuming and often costly activity that will disrupt clinical duties. A competent facilitator is needed to design and prepare a scenario, and the crew members need time for training and debriefing.

Lack of assessment

Similar to our findings on simulation-based training, physicians undergo NTS assessment significantly less often than the other professional groups. Domain-specific NTSs have been identified, and assessment tools have been developed, for teams and individuals in medical teams, but not in the context of prehospital critical care²² and HEMS. Without a frame of reference, the description and evaluation of NTSs will be ambiguous. What is assessed, how it is assessed and how this information is used will vary—and ultimately training may not be assessed at all. This may well be the reason for the lack of assessment in our data. Without carrying out thorough evaluations, it can be difficult to test skills, to provide feedback on skill development, to point out strengths and identify training needs, and to determine whether an NTS training programme (CRM) is effective at improving the skills in question.¹²

Teamwork

Single- and multi-disciplinary team training are complementary methods, and personnel should participate in both to develop teamwork skills.¹⁰ Our data imply that not all simulation training takes place within the framework of a complete multidisciplinary HEMS crew.

Team performance may directly affect patient safety.^{10 16} A shared understanding—a shared mental model—of the task in hand and of the other team members’ roles has been identified as one important characteristic of a high-performance team.^{10 12} The physician and the HCM are primarily responsible for providing patient care on-scene. The pilot is the only crew member with no formal medical skill competencies, and he is thus least qualified to take part in the medical treatment. However, the pilot is often involved in simpler patient-related tasks to assist the medical crew, such as checking of medical equipment, resuscitation and preparing the patient for transport on the stretcher. This is similar to the physician, who does not have formal flight training, but has responsibilities related to flight safety both during take-off and landing and in-flight in order to supplement the pilot and the HCM. These tasks require teamwork and understanding across disciplines.

Duty hours

Regularly scheduled on-call duty for Norwegian HCMs lasts for up to 7 consecutive days around the clock. This is much longer than similar rotor-wing air medical programmes in the USA, where the maximum shift length has been reported to be 48 h.²³ A high number of duty hours a week is common among emergency medical service providers, and has been suggested to be in part culturally determined. Long shifts and on-call working is recognised as a risk to patients and operational safety.^{23 24} The workload and frequency of HEMS missions will vary during the on-call period. On-duty rest and sleep is permitted for all crew members in Norway and must be obtained between missions. Working at night, for irregular hours, is inevitable and results in disrupted sleep and a displaced sleep schedule, which might affect mental performance, health and the risk of adverse events.^{23 24} To prevent fatigue, pilots and HCMs are protected by flight time limitations and rest time rules,²⁵ but these regulations do not deal with quality of rest and sleep between missions. HEMS physicians in Norway are protected by the same rest and sleep regulations as pilots and HCMs, but the regulations are enforced differently in different HEMS bases. Crew members in our study reported that they receive limited training on how to recognise and cope with fatigue.

Strengths and limitations of the study

The response rate for the survey was 81.8%, with few data missing, which is considered satisfactory and is a strength of the study. The study is limited by its small sample size. Despite a high response rate, which increases effective sample size and reduces non-responder bias, the number of respondents was too small to detect statistically significant differences between all the professional groups.

The study was limited to a set of seven broad generic NTS categories claimed to have general applicability across a wide range of high-risk work settings.¹² The questions did not differentiate between composite team assessment and individual performance assessment.¹²

We did not include conceptual explanations and definitions in our questionnaire. We assumed that the crew members already shared a common vocabulary for discussing the basic principles of NTSs, since CRM training is mandatory for all crew members in Norway.

In order to achieve maximum response rates, both a paper version and an electronic version of the questionnaire were made available at the same time in the data-collection period. We emphasised that each respondent had to fill out only one form each, either paper or electronic. We considered the likelihood of multiple responses from one individual very small and the advantage of a high response rate correspondingly large. There is still a possibility that a single respondent may have filled out more than one form.

The questionnaire was anonymous, and responding to the questionnaire was voluntary. However, there is a possibility that respondents to this type of questionnaire do not respond truthfully, or do not remember details exactly. This may result in under- or over-reporting. We do not have any information about the non-responders.

Implications

This study has implications for current practice and future research. Existing training requirements, and assessment criteria, for Norwegian HCMs are based on generalised statements of performance outputs. They do not clearly specify how often training for and assessment of NTSs should be. Mandatory NTS training requirements for crew members in the civilian Norwegian HEMS need to be specified as an incentive to train, with a view to licensing and registration. Special emphasis needs to be placed on patient safety issues relating to fatigue and sleep homoeostasis among crew members in HEMS. Future research might explore how to increase frequency of simulation-based NTS training with minimal disruption to clinical duties and with little expense.