Measuring non-technical skills in medical emergency care: a review of assessment measures

Abstract

Aim

To review the literature on non-technical skills and assessment methods relevant to emergency care.

Background

Non-technical skills (NTS) include leadership, teamwork, decision making and situation awareness, all of which have an impact on healthcare outcomes. Significant concerns have been raised about the rates of adverse medical events, many of which are attributed to NTS failures.

Methods

Ovid, Medline, ProQUEST, PsycINFO and specialty websites were searched for NTS measures using applicable access strategies, inclusion and exclusion criteria. Publications identified were assessed for relevance.

Results

A range of non-technical skill measures relevant to emergency care was identified: leadership (n = 5), teamwork (n = 7), personality/behavior (n = 3) and situation awareness tools (n = 1). Of these, 9 have been used with emergency care populations/clinicians. All had varying degrees of reliability and validity. In the last decade there has been some development of teamwork measures specific to emergency care with a predominantly global and collective rating of broad skills.

Conclusion

A variety of non-technical skill measures are available; only a few have been used in the emergency care arena. There is a need for an increase in the focused assessment of teamwork skills for a greater understanding of team performance to enhance patient safety in medical emergency care.

Background

The term non-technical skills (NTS), sometimes referred to as ‘soft’ skills or crew (sometimes ‘crisis’) resource management is derived primarily from the aircraft industry. Such skills include leadership, teamwork, decision making and situation awareness.¹ There is a growing body of evidence that team functioning has an impact on healthcare outcomes^2–5 and the mental health of employees.² Teamwork behaviors need to be learned for patient safety⁶ and multi-professional education and team training can be utilized to improve team effectiveness.⁷

To lead is derived from the Latin as a path or goal and a leadership is defined as an individual who “influences others in the group more than they themselves are influenced”,⁸ whilst key characteristics and components of teamwork include clarity of goals, applicable communication, defined roles and horizontal leadership.⁶ The benefits of a flat hierarchy and open access to leaders are evident in patient safety reviews;^9,10 however leadership is contingent on the needs of the situation and may require a command and control form in time bound emergency situations.

Situation awareness (SA) is a concept developed for aircraft cockpit crews¹¹ which has been measured in anesthesia and surgery.^11,12 It is described as the “perception of the elements in the environment within a volume of space and time, the comprehension of their meaning and the projection of their status in the near future”.¹³ These skills are all encapsulated in the study of human factors; “the environmental, organizational, and job factors, and human individual characteristics which influence people at work”.¹⁴

It is quite possible that teams may behave in different ways but with the same outcome, a result that Katz and Kahn¹⁵ call equifinality or equal finality. Alternatively, Tyler¹⁶ suggests that there are multiple possibilities, highlighting the fact that similar situations can generate completely different outcomes. For these reasons it is often inappropriate to designate hard outcome measures of performance.¹⁷ Observation of behavior related to outcome is more applicable, especially in critical situations where procedures are clearly defined and can be measured relatively easily. This is the case for resuscitation teams, where it would be inappropriate to measure survival rates in relation to team or leader performance alone, because survival is also dependent on factors such as the primary heart rhythm and underlying disease.¹⁸ For these reasons NTS should be judged in relation to the context.

There are certain core principles for development and use of teamwork assessment tools; for example, commencing with an appropriate theory, being aware that teams and situations change and using observational methods with a valid and reliable instrument.¹⁹ However, observational techniques are limited by the accuracy of information recording which has led to the use of video recording of trauma²⁰ and cardiac resuscitations,^21,22 where such methods are particularly useful in identifying the coordination and cooperation of a team. However, in the United Kingdom severe ethical restrictions relating to requirements for retrospective patient approval for such projects have reduced the frequency of such studies.

There are also a number of disadvantages to direct observational techniques, one of which is reactivity and distortion of behavior due to the intrusive presence of an observer/camera; in essence the Hawthorne effect – a change in behavior due to direct observation.²³ However Niebuhr and colleagues²⁴ counter this, suggesting that most people are used to being observed in their day to day work, and that emergency teams are likely to be highly focused on the task and less distracted by an observer.

Significant concerns have been raised about the rates of adverse medical events, many of which are attributed to NTS failures^1,25,26 and often exacerbated by the transient individual nature of medical work. In response a variety of NTS assessment tools and training schemes have been devised across the developed world. These can take the form of self and subordinate ratings of performance; however observational ratings of team performance in simulated and live settings, with applicable feedback and training, are generally considered to be more rigorous.²⁴

Simulation techniques are fast becoming the preferred teaching method for improving a variety of non-technical skills²⁷ including emergency skills^28,29 teamwork³⁰ and decision making³¹ with advantages over standard forms of life support training.³²

The aim of this paper is to review the literature on non-technical skills assessment methods relevant to the emergency department (ED) and emergency care, in order to address the following questions: 1. How are non-technical skills (NTS) defined in the context of emergency care? 2. What NTS measures are applicable to emergency care? 3. Which NTS measures have been used in emergency care?

Design

A search of the literature was conducted to locate and review instruments to quantify non-technical skills including: teamwork, leadership, decision making and situation awareness. Electronic databases were searched from 1996 to 2009 and included Ovid Medline, ProQUEST, PsycINFO and specialty websites, for example, the National Patient Safety Association UK and Resuscitation Council UK. Access strategies included the keywords: leadership, teamwork, situation awareness, non-technical skills, standards, task performance analysis, resuscitation, and medical emergency team, together with author and journal searches. In addition to this, systematic searches of the teamwork literature, individually performed by the authors, were accessed for key papers between 1960 and 1996.

Inclusion/exclusion criteria

Papers were included that reported studies: a) Focusing on non-technical skills measures applicable to the emergency care environment. b) That were undertaken in any country. Papers were excluded if they were: a) Not available in English. b) Did not include, or provide access to the measurement tool. c) Related to technical or clinical skills measures alone. Following this, primary search articles not meeting any of the inclusion criteria by title or abstract were excluded; remaining abstracts and full papers were reviewed by 2 authors (Figure 1).

Figure 1.

Flow diagram of review process and outcomes.

Abbreviation: NTS, non-technical skills.

Search outcome

A range of papers relevant to emergency care were identified that included the following measures; leadership (n = 5), teamwork (n = 7), personality/behavior (n = 3) and situation awareness tools (n = 1). Of these, 9 have been used with emergency care populations/clinicians.

Results

General NTS measures

From our search we identified a wide range of instruments for the measurement of leadership, team behavior and personality which may be beneficial in emergency care environments.

Leadership and team rating

Campbell’s Leadership Potential Index Self versus Other³³ measures leadership potential. Leaders rate themselves using 160 adjectives, whilst subordinates use the same list to describe their leader. The key orientations are leadership, creativity, physical energy, productivity, likeability, and psychological comfort (for example, confidence/suspicion).

The Leadership Practices Inventory³⁴ measures five leadership practices which can be used for self rating or for third party rating of leaders, these included; challenging the process, inspiring a shared vision, enabling others to act, modeling the way, and encouraging the heart. Importantly a leadership effectiveness scale is included which correlates with the leadership practices inventory.

Team Excellence (TE) and Team Excellence Lead³⁵ were developed for rating teams and leaders using a grounded theory approach and identified team strengths as: goal clarity, competence, and standards of excellence. Weaknesses were identified as; poor commitment, lack of external support and poor collaboration. The final leadership scale includes 13 leadership behaviors such as; articulating goals, avoiding excessive priorities, supporting team members, confronting inadequate performance, and being open to new ideas.

Kolb³⁶ combined the Team Excellence-Lead instrument and the Leader Behavior Description Questionnaire (LBDQ), Form X11³⁷ which had been found to be both valid and reliable.^{34,35,38–40} The resultant scale retained many of the team excellence items but also combined terms to create categories such as, provides autonomy, and added categories such as personal/professional qualities. In the medical field this tool has been used to evaluate outcomes from the National Health Service (NHS) leadership development programme Leading Empowered Organizations (LEO).⁴¹

Personality and behavior tests

As a useful measure of team working potential and decision making, the most common personality test is the Myers Briggs Type Indicator (MBTI) which is administered to over 3.5 million people per annum.^42–45 The measure is based on Jung’s theory of psychological types which describes a focus on the external world of people, things and experience (extroversion) and the internal world of inner processes and reflections (introversion).

Of potential use to medical team development is Catell’s 16 Personality Factors Test (16PF),⁴⁶ which identifies a collection of behaviors that characterize a personality type, but also includes a reasoning ability measure.

Measuring personality is informative, although limited, in that personality becomes fixed at an early age. More useful perhaps is a measure of preferred behavior which enables both individual and team insight.

Fundamental Interpersonal Relations Orientation – Behavior (FIRO-B)⁴⁷ is one such tool which measures areas of interpersonal needs (both expressed and wanted) identified as inclusion, control and affection. Behavioral preferences can also be mapped as a team profile.

All of these tools have potential in the emergency care field depending the needs of the ED. Some can be used as measures of potential or of individual, leadership or team performance that may be rated by self, subordinates or peers, through retrospective or direct ratings of observational performance. When selecting an instrument it is not only important to consider the validity, reliability and feasibility of the measure but the foci required. For example, the Leadership Practices Inventory incorporates categories such as ‘modeling the way’ and ‘encouraging the heart’ which may be too broad a categorization for the intimate analysis of emergency behavior, whilst Team Excellence and the Leader Behavior Description Questionnaire (LBDQ) use a more focused micro approach to the measurement of team behaviors.

NTS measurement in emergency care

More specifically and as listed in Table 1, we focused on NTS measures that have been designed or used in emergency care.

Table 1.

Non-technical skill measures

Study	Target group (n =)	Assessment measures	Response scale	Measurement development	Validity and reliability	Outcome
Adapted Leadership Behaviour Description Questionnaire (Initiating Structure)48	Resuscitation teams (n = 20)	An observational rating scale for 9 leadership items	Always (4) Very often (3) About as often as not (2) Seldom (1) Never (0)	Extensive development by Stogdill.37 Refined and adapted from 20 video recorded resuscitation events	Extensive prior testing and in the adapted form good uni-dimensional validity and inter-observer agreement (Cohen’s kappa 0.71)	A measurement of resuscitation team leaders performance based upon their command and control ability
Emergency Team Dynamics56	Emergency teams and ECPs (n = 20 teams and 24 ECPs)	Eight items, rating leadership and teamwork	Always (4) Very often (3) About as often as not (2) Seldom (1) Never (0)	Developed with 20 resuscitation teams48 and 24 emergency care practitioners	Good, face, content and uni-dimensional validity and inter-observer agreement (Cohen’s kappa > 0.82)	A measurement of emergency team performance
Communication Competency Questionnaire56,57	Emergency Care Practitioners (ECPs) (n = 24)	Communication	A scale of 0–6 from disagreement to agreement	Extensive development by Monge et al57 Refinement for ECPs	Good, face, content and one-dimensional validity and inter-observer agreement (Cohen’s kappa > 0.82)	A measurement of communication competence in emergency care
Mayo High Performance Teamwork Scale58	High performance multi-professional medical teams (n = 107)	16 items covering teamwork, leadership, communication	0 = never or rarely 1 = inconsistently 2 = consistently	107 participants at a crisis resource management (CRM) course generated an initial 273 ratings of key CRM skills	Satisfactory internal consistency (alpha 0.85) and construct validity based on person reliability (0.77), person separation (1.85), item reliability (0.96) and item separation (5.04). Sensitivity to change – pre-post training	A measure of crisis teamwork skills designed for simulation based reflection and training
Technical and non-technical rating scale for septic shock59	Physicians (individuals and teams) managing septic shock (n = 23)	Technical and non-technical skill ratings the later measuring 6 items covering planning, communication, task distribution and information use	5-point rating scale with rating guidance given for ratings of 1, 3, and 5	Non-technical ratings developed by authors. 23 video records of simulated septic shock events reviewed by 2 physicians	Expected inter-group variability Inter-rater reliability correlation r = 0.88 Correlation with technical skills r = 0.40	A measure of individual and teamwork technical and non-technical skills for the management of septic shock. Used for benchmarking and intervention application
Ottawa CRM Global Rating Scale30	Medical students (n = 60)	Six items measuring leadership, problem solving, situation awareness, resource utilization and communication skills	7 point rating scale with rating guidance given for ratings of 1,3,5,7	Construct evaluation of previously developed instrument. V ideo rating by three assessors of simulation based medical CRM skills for 32 1st year and 28 3rd year medical residents	Good construct validity based on content, response process, internal structure and relation- ship to other variables. Inter-observer agreement correlation 0.66	A broad measure of teamwork skills specific to the management of the critically ill
Trauma Team Evaluation Tool60	Trauma teams (n = 10 teams of 3)	Technical and non-technical skill ratings the later covering 5 items measuring leadership, team roles, communication, patient assessment and handling of distractions	3 point scale – 0 to 2 with applicable descriptors	Not described	Not tested	A measurement instrument primarily designed for the measurement of technical trauma resuscitation skills. Limited coverage of non-technical skills
Situation Awareness Global Assessment Techniques (software for the development of assessment items)62	Military, power, aviation, air traffic control, medical	Situation awareness – SA (perception, understanding and prediction)	Nominal – yes/no responses	Questions are developed for each context	Objective direct measurement of SA. Previously assessed validity and reliability64	Measurement of situation awareness bases on respondents real time ratings
Team Climate Inventory67	Healthcare teams (n = 148 teams)	Team climate including group climate, participative safety, vision, task orientation and support for innovation	44 items; 26 statements about communication and innovation with a 5 point Likert scale (strongly disagree to strongly agree) and 18 questions about team objectives and task style with a 5 point Likert scale (not at all to completely)	Studies of 148 teams in healthcare and industrial settings	Extensive testing by Anderson and West67	A measurement of team climate applicable to healthcare settings

Abbreviations: ECP, emergency care practitioners; SA, situation awareness; CRM, crisis resource management.

Leadership and teamwork

In a study examining resuscitation leadership performance⁴⁸ the LBDQ,^49–51 later developed into LBDQ (Form XII),^37,51,52 was further adapted. This widely evaluated measure of leadership behavior^52–54 includes 2 factors to describe leadership behavior; consideration and initiating structure. Consideration is the extent to which leaders show consideration towards members of the team which is deemed to be relatively unimportant when studying emergency situations as there is limited time to build relationships. Initiating structure is the extent to which a leader manages the structural aspects of the team and includes elements such as detailing what and how things should be done with clear command and control structures. This focus was considered to be applicable to resuscitation leadership performance and was adapted into the form shown in Table 2. The degree to which the leader spent time actively performing tasks that could otherwise have been delegated was measured and, based on an in-depth review of the literature, a measure of team function was produced. Both scales were found to have excellent uni-dimensional validity and inter-observer agreement ratings (Cohen’s kappa) of greater than 71%. Finally, based on performance criteria set by the Resuscitation Council (UK) resuscitation task performance was also measured.

Table 2.

Adapted LBDQ (Form XII): LBDQ (Initiating Structure)^a

1. The leader let the team know what was expected of them (through direction and command)

2. The leader demonstrated the use of uniform guidelines

3. The leader displayed a positive attitude

4. The leader decided what should be done

5. The leader decided how things should be done

6. The leader assigned group members to particular tasks

7. The leader made sure that his/her part in the team was understood by the team members

8. The team leader planned the work to be done

9. The team leader maintained definite standards of performance

Notes:

^aItems scored using the following rating (score): A, Always (4); B, Very often (3); C, About as often as not (2); D, Seldom (1); E, Never (0).

Cooper and Wakelam 1999.⁴⁸

Abbreviation: LBDQ, Leadership Behavior Description Questionnaire.

The results indicated high correlations between structured team leadership, enhanced team dynamics and improved task performance, whilst team outcomes were reduced where leaders actively participated. In the opinion of the researchers highly structured teams managed through command and control structures would enhance resuscitation performance.⁴⁸

The LBDQ has also been used for the evaluation of a leadership development seminar introduced into an advanced life support course.⁵⁵ With an additional ‘hands off’ item, observational ratings of leadership behavior were performed in simulated resuscitation attempts at the end of the course. The LBDQ therefore proved beneficial for the rating and feedback of leadership performance in both live and simulated settings.

In a different context, the collaborative practices of Emergency Care Practitioners (ECP) working in a UK ambulance service were examined.⁵⁶ The LBDQ was again used but on this occasion quantifying consideration and initiating structure, whilst the Team Dynamics scale was updated and retitled as Emergency Team Dynamics (ETD) (Table 3). In addition, the Communication Competence Questionnaire (CCQ)⁵⁷ was used for rating communication skills (Table 4). LBDQ and CCQ were cited as having a high validity and reliability^48,51,55,57 and in this study all three scales, including ETD, demonstrated good face and content validity, assessed by the research team and two external experts. The internal reliability/consistency (Cronbach’s alpha) for all the scales was >82%. The results indicated a positive correlation between leadership, communication and teamwork and that the higher grade and more highly educated emergency care practitioners (ECPs) were found to be the more effective leaders.⁵⁶

Table 3.

Emergency team dynamics^a

1a.* The leader let the team know what was expected of them (through direction and command)

1b. The team transferred information (communication skills)

2. The team were adaptable (within the roles of their profession)

3. The team were co-ordinated

4. The team co-operated

5. The team used initiative

6. The team put effort into its work

7. The team had a positive spirit and morale

Notes:

^aItems scored as 0, never; 1, seldom; 2, about as often as not; 3, very often; 4, always.

^*Item used only where the Leadership Behavior Description Questionnaire is not used. Cooper, et al.⁵⁶

Table 4.

Communication competence questionnaire

1. The ECP has good command of the language

2. The ECP medicalises language appropriately

3. The ECP is sensitive to others needs

4. The ECP typically gets right to the point

5. The ECP pays attention to what other people say to him/her

6. The ECP deals with others effectively

7. The ECP is a good listener

8. The ECPs writing is understandable

9. The ECP expresses his/her ideas clearly

10. The ECP is understandable when he/she speaks

11. The ECP generally says the right thing at the right time

12. The ECP is easy to talk to

13. The ECP usually responds to messages quickly (phone calls, emails etc)

Notes: Rated using the following scale: 6, very strong agreement; 5, strong agreement; 4, mild agreement; 3, neutral feelings or don’t know; 2, mild disagreement; 1, strong disagreement; 0, very strong disagreement.

Abbreviation: ECP, emergency care practitioner.

Adapted from Monge et al⁵⁷ in Cooper et al 2007.⁵⁶

The Mayo high performance teamwork scale⁵⁸ was developed from 107 participants’ ratings of key crisis resource management (CRM) skills during CRM training in a simulation center. The final measure included 16 items rating teamwork, leadership and communication on a three point scale. Extensive validity and reliability testing included satisfactory internal consistency (Cronbach’s alpha 0.85), and construct validity based on person reliability (0.77), person separation (1.85), item reliability (0.96) and item separation (5.04). In addition sensitivity to change was measured by assessing changes to performance before and after CRM training. The measure takes a broad overview of crises (ie, it is not context specific) that could be useful in crisis resource management training for participant feedback.

Ottestad and colleagues⁵⁹ developed a technical and non-technical rating scale for individuals and teams managing septic shock. The non-technical rating scale was developed by the authors and included 6 items covering planning, communication, task distribution and information use. The resultant scale was used by 2 of the authors (physicians) to rate 23 video recordings of simulated septic shock scenarios managed by a range of medical grades. The outcomes demonstrated an expected inter-group variability, inter-rater correlation of r = 0.88 and a significant but low correlation with technical skills ratings (r = 0.40). The measure should be used for the rating of teams managing septic shock.

The Ottawa Crisis Resource Management Global Rating Scale³⁰ was used to evaluate simulation-based performance of medical teams managing the critically ill. The instrument is based upon 6 items that measure leadership, problem solving, situation awareness, resource utilization and communication, on a 7 point rating scale. Based upon a total of 61 video ratings by 3 observers of simulated critically ill patients, the instrument was found to have good construct validity but a relatively low inter-observer agreement correlation (0.66). The instrument provides a broad measure of teamwork skills specific to the management of the critically ill. However, further work is required to demonstrate reliability.

The Trauma Team Evaluation Tool⁶⁰ was included in a study of military trauma resuscitation teams, which aimed to assess the feasibility of evaluating skills in a simulated environment. The performance of 10 three-person multi-professional teams was compared to 5 expert teams using the evaluation tool. Designed as a predominantly technical rating scale including airway, breathing, circulation and disability assessments, the tool also includes measurements of leadership, team roles, communication, patient assessment and handling of distractions. Development of the measurement and its validity and reliability is not described.

Situation awareness

Originally developed from aircraft industry work on cockpit crew resource management and later developed in anesthesia, the term Situation Awareness (SA) is formally defined by Endsley as a perception of the state of the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future.¹³ To improve performance in emergency care, Lynch and Cole⁶¹ highlight the importance of high situation awareness or seeing the bigger picture; an objective that can be achieved by regular briefings and patient handover.

The 3 levels of SA (perception, understanding and prediction) can be mapped and measured in a simulated environment using self or observer ratings, or previously validated software known as Situation Awareness Global Assessment Technique (SAGAT).⁶² As it does not rely on self or observer ratings the SAGAT is considered to be the more objective form of measurement⁶³ and is a reliable and valid assessment tool which may enhance curricula and patient management.⁶⁴

Situation awareness questions are developed to assess the perception, comprehension and projection elements of the situation using Goal Directed Task Analysis (GDTA). Experts identify the goals and sub-goals associated with a particular work task and the decisions required to achieve these goals, producing questions that cover the three levels of SA.⁶³ Simulation exercises are then halted (called a freeze) at a random point and participants asked the standardized SA questions about their perception of the situation. The freeze is normally set after the first 3 minutes of each scenario to allow participant to build up a picture of the situation.

Hogan and colleagues⁶⁴ used the SAGAT during the assessment of trauma skills and Kinsman and colleagues⁶⁵ measured situation awareness using the SAGAT in a simulation based study which was aimed at identifying the factors that influenced student nurses’ ability to identify the deteriorating patient. Situation awareness questions were judged to have good face validity, ratings revealed an expected average performance considering students’ education and experience, and there was no difference in the mean scores of participants tested by each of the two SA facilitators.

Evidence from anesthesia¹¹ and surgery⁶⁶ suggests that an increased awareness of a situation will enhance patient safety and improve medical staff performance. It is likely that repeated high stakes, high fidelity simulation will help to enhance practitioners’ performance in this domain.

Team climate inventory

After studying 148 teams from a range of healthcare and industrial settings Anderson and West developed a measure of team functioning; the Team Climate Inventory (TCI).⁶⁷ This instrument has been used in a range of healthcare settings across a number of countries.^68–70 TCI refers to the manner of working together that the team has evolved⁶⁷ and includes 5 categories: group climate, participative safety, vision, task orientation, and support for innovation. Participative safety is the degree to which team members are threatened or insecure, covering for example, information sharing, safety and interaction. Vision is the clarity of the aims and objectives of the group and includes clarity and perceived value. Task orientation is a measure of commitment to excellence and includes ratings of excellence, appraisal and tasks. In the final computer generated report, team responses are compared with those of a similar team. The TCI can be used as a pre- or post- outcome measure in interventional studies, or as a demographic variable/diagnostic tool^69–71 administered on a single occasion, for observational studies.

For example, in consultancy work the authors (RE and SC) were contracted to review patient pathways and performance in an emergency department. The work included quantitative and qualitative measures of performance including; a patient audit, observation and interviews. Team work was identified as a key issue in the performance of the department and TCI was therefore introduced as a measure of performance. From the findings senior management were able to make a number of changes to team membership, to highlight the team climate issues and to introduce training interventions.

Discussion

Non-technical skills are predominantly defined as teamwork skills, which include leadership, decision making and situation awareness. This begs the question as to why the term non-technical skills is used in the first place? In the 21st century kudos is retained for technical skills and ability, implying that non-technical skills are second class. However, it is evident from a number of patient safety reports^9,10 that teamwork skills and communication are essential aspects of patient safety. The term teamwork skills would be a truer, clearer and less pretentious description of an essential aspect of health care.

In the emergency care field teamwork attributes demand a flexibility that may not be apparent in other fields. For example, the day to day management of an emergency department or ambulance service will be quite different to the demands of leading a resuscitation team or any other time bound emergency situation. This, coupled with the transient nature of employment, ad hoc teams, and dynamic situations, means that an understanding and monitoring of applicable teamwork is essential.

A wide range of applicable measures are available all with varying degrees of reliability and validity. Many focus on leadership^34–36 and most are quite dated, whilst a few focus on contemporary measures of personality^42–46 and preferred group behavior⁴⁷ which may be useful in recruitment and team allocation. In the last decade there has been some development of teamwork measures specific to emergency care covering, for example, leadership,⁴⁸ teamwork^{30,56,59,60,67} and situation awareness.^64,65 All tend to focus on the more global collective interactive skills leaving room for a focus on individual clinical decisions, for example Kinsman and colleagues⁶⁵ focus on missed clinical cues during patient deterioration episodes. Few of the measures identified have been used on multiple occasions in emergency care, suggesting that there are differing views about the relative dimensions of non-technical skills, and/or a lack of emphasis on their importance.

With increasing concerns about patient safety and the need for multi-professional teamwork there is a need for focused, context specific, feasible measures of emergency performance. For example, Cooper and colleagues⁷² further refined their work with the Leadership Behavior Description Questionnaire, Emergency Team Dynamics and situation awareness to produce a single tool for the observational assessment of NTS performance in resuscitation teams. The tool was found to be valid and reliable after an in depth review of the literature, expert development of items, independent assessment for construct validity, pre-testing on 56 real and simulated resuscitation attempts and pilot testing on 15 simulated events. Content validity, inter-rater agreement and test-retest ratings were all at an acceptable level and the tool was found to be feasible in the observational setting. The final 12 item Team Emergency Assessment Measure (TEAM) is likely to be used in live and simulated resuscitation attempts for the rating and feedback of team performance.

Conclusion

A variety of valid and reliable non-technical skill measures are available; however only a few have been used in the emergency care arena. There is a need for an increase in the focused assessment of teamwork skills to enable a greater understanding of team performance and to enhance patient safety in the emergency care field.