Conflict resolution in anaesthesia: systematic review

Abstract

Background

Conflict is a significant and recurrent problem in most modern healthcare systems. Given its ubiquity, effective techniques to manage or resolve conflict safely are required.

Objective

This review focuses on conflict resolution interventions for improvement of patient safety through understanding and applying/teaching conflict resolution skills that critically depend on communication and improvement of staff members’ ability to voice their concerns.

Methods

We used the Population-Intervention-Comparator-Outcome model to outline our methodology. Relevant English language sources for both published and unpublished papers up to February 2018 were sourced across five electronic databases: the Cochrane Library, EMBASE, MEDLINE, SCOPUS and Web of Science.

Results

After removal of duplicates, 1485 studies were screened. Six articles met the inclusion criteria with a total sample size of 286 healthcare worker participants. Three training programmes were identified among the included studies: (A) crisis resource management training; (B) the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) training; and (C) the two-challenge rule (a component of TeamSTEPPS), and two studies manipulating wider team behaviours. Outcomes reported included participant reaction and observer rating of conflict resolution, speaking up or advocacy-inquiry behaviours. Study results were inconsistent in showing benefits of interventions.

Conclusion

The evidence for training to improve conflict resolution in the clinical environment is sparse. Novel methods that seek to influence wider team behaviours may complement traditional interventions directed at individuals.

Introduction

Perioperative care is complex and is delivered by teams and individuals with diverse training, experience and personality. Successful outcomes in healthcare are increasingly understood to be the product of the team as a whole rather than any single member.¹ Healthcare is almost exclusively delivered by teams comprising two or more actors of differing specialties. Team members have to cooperate in an effective way and to share their knowledge to achieve a common goal,¹ but to do so, they require an environment with an atmosphere of open and clear speaking up.

Ensuring patient safety in such settings can be challenging, particularly when clinical work is unpredictable and requires rapid assessment and decision-making, urgent intervention and effective speaking up.² The operating room is an exemplar environment for these issues—multiple team members with sometimes conflicting organisational and patient-specific goals, in a dynamic environment, with limited ability for individuals to remove themselves in time or place from conflict situations. Inevitably conflicts arise between team members; these may be straightforward such as one individual not having the same information as another, or more complex such as differing interpretations of the same information, conflicting goals or ‘personality clashes’. Unresolved conflict is detrimental to patient safety in the short term—through impaired teamwork. It may also have longer term through effects on staff morale and turnover.³ However, healthcare professionals have little, if any, formal training in conflict resolution, and there is an increasing body of evidence asserting that poor conflict resolution skills can have an adverse influence on patient outcomes (patient safety) and job satisfaction^{1 2} and team performance. Conflict has been defined as ‘a state of disharmony between incompatible persons, ideas, or interests’, and has been observed to be a significant and recurrent problem in most modern healthcare systems.² There are various types of conflict, with the most common being interpersonal conflict.

The General Medical Council⁴ and the Accreditation Council for Graduate Medical Education have highlighted the importance of conflict resolution skills and other speaking up skills in resident training.⁵ Such skills are taught in other industries such as aviation and business,² wherein conflict resolution and teamwork principles have been shown to be the bases of teaching and improving safety outcomes.²

There are a number of potential interventions that can be undertaken in operative settings to improve conflict resolution. To our knowledge, there has been no systematic review of interventions that can improve the ability of staff to resolve conflict within the operating theatre. This review focuses on conflict resolution interventions and strategies and their impact on educational, behavioural and patient-level outcomes.

Methods

We used the Population-Intervention-Comparator-Outcome model as an outline for our study.⁶ This approach is intended to provide some structure to the search process at the outset, allowing relatively objective assessment of whether studies are eligible for inclusion or not. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist⁷ is found in online supplementary appendix 1.

Population

Healthcare professionals (anaesthetists, surgeons and trainees, anaesthesia assistants, nurse-anaesthetists) in any theatre setting.

Intervention

Undergoing one or more speaking up training and conflict resolution programmes or the use of a professional/social context to influence conflict/speaking up behaviours.

Comparator

Healthcare professionals who have not undergone any specific speaking up training during the study period.

Outcomes

Any measured conflict resolution outcomes. The Kirkpatrick categories of outcome following learning,⁸ which promptly provides identifiable and easy to measure outcomes in learners,⁹ was used as the underlying framework. This approach provides a hierarchy of potential outcomes from learner reaction through to ‘real-world’ changes in patient or organisational outcomes.

In this study, speaking up was defined as ‘speaking up with one’s, and/or other team members doubts, differing opinions or potential problems about decision or course of action in medical care.’¹⁰ This definition includes non-verbal communication.

We retrieved relevant papers, both published and unpublished, up to February 2018 from the following databases: Cochrane Library, EMBASE (via Ovid platform), MEDLINE (via Ovid platform), SCOPUS and Web of Science. A combination of MeSH and free text terms was used to increase the likelihood of identifying relevant studies. The following search terms were used: (operating room* OR anaesthesia* OR anaesthetist* OR trainee* OR operating theatre* OR theatre intensive care*) AND (communication* OR communication barrier* OR communication failure* OR team work* OR decision making* OR teaching* OR education* OR authority* OR superior* OR safety*) AND (conflict* OR challenge* OR speak* OR speak up* OR and speak out* OR resolution*). They were initially run individually and then were combined in each database. No limits (publication date, journal of publication or language) were applied. Hand searches were also conducted in relevant journals on patient safety and organisational research (online supplementary appendix 2).

Inclusion criteria

All papers considered had to have a primary or clear secondary focus on conflict resolution and techniques or contexts to improve staff members’ ability to voice their concerns and patient safety in the operating theatre.

Exclusion criteria

We excluded studies that did not investigate any intervention (eg, teaching programme or social context) and that investigated the effects of conflict resolution and speaking up in general medical practice (ie, not the operating theatre).

Selection of studies

Two authors—DSA and TCM—independently screened the titles, abstracts and full texts of retrieved studies to identify potentially relevant articles for inclusion.

Data extraction

Data extraction was performed independently by DSA and TCM. A standardised data collection form was designed to collect specific study details including sample size, type of conflict, factors influencing conflict, barriers to conflict resolution, and data on the training programme, period of training and outcomes. Studies were included in the review based on eligibility criteria. All data were independently verified by the senior author (IKM).

The risk of bias was assessed using the Grading of Recommendations Assessment, Development and Evaluation tool¹¹ and the Quality Assessment Tool for Before-After (Pre-Post) Studies With No Control Group.¹²

Results

Overview

The initial online search produced 3830 papers; after removal of duplicates, we screened 1485 studies by title yielding 11 potential eligible studies; for full text assessment. Six studies with a total sample size of 286 participants were deemed eligible for inclusion in this review,^13–18 five were from initial online search and one was from hand searching of reference lists of included papers. No additional papers were obtained through forward searching. All of the studies in this review were conducted in the USA and Canada. The flow chart of study selection is shown in detail in figure 1.

Figure 1.

Flow diagram of included and excluded studies.

Training programmes

Three distinct training programmes were identified among the included studies: (A) crisis resource management (CRM) training;¹³ (B) the Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) training;¹⁴ and (C) the two-challenge rule^{15 16} which is a component of TeamSTEPPS. Two studies^{15 16} used a social construct (behaviour of a senior clinician) as a potential-modifying mechanism. Key outcome measures across all programmes included improvement in speaking up, patient safety and positive assertiveness in challenging hierarchy. Characteristics of the included studies are outlined in table 1, and the main outcomes of each training programme are outlined in table 2.

Table 1.

Characteristics of included studies

Study	Country	Department and participants (n)	Type of study	Intervention	Duration	Kirkpatrick level outcomes
						K-1	K-2	K-3	K-4
Friedman et al 13	Canada	Anaesthetist residents (50)	Simulation-randomised	CRM	1-hour session	Kirkpatrick: level 1	Kirkpatrick: level 2
Sawyer et al 14	USA	Physicians, nurses and respiratory therapists (42)	Clinical-simulation	TeamSTEPPS	6 hours		Kirkpatrick: level 2
Pian-Smith et al 15	USA	Trainee anaesthetists (40)	Clinical-simulation	Two-challenge rule	30–45 min workshop		Kirkpatrick: level 2
Raemer et al 16	USA	Non-trainee anaesthetists (71)	Simulation-randomised	Two-challenge rule	50 min workshop	Kirkpatrick: level 1	Kirkpatrick: level 2
Sydor et al 17	Canada	Trainee anaesthetists (49)	Simulation-randomised	Team behaviours/team hierarchy	–	Kirkpatrick: level 1	Kirkpatrick: level 2	Level 3: behaviour
Friedman et al 18	Canada	Trainee anaesthetists (34)	Simulation-randomised	Team behaviours/team hierarchy	–	Kirkpatrick: level 1	Kirkpatrick: level 2	Level 3: behaviour

CRM, crisis resource management; K, Kirkpatrick levels; TeamSTEPPS, Team Strategies and Tools to Enhance Performance and Patient Safety; –, no data available.

Table 2.

Summary of main outcomes for each training programme

Training programme	Main assumption	Taught principles/domains	Strengths	Weaknesses
CRM	Healthcare practitioners to voice their concern (‘speak up’) when safety concerns arise. Decision-making as a teamwork principle. Emphasis on cross-discipline understanding assuring all team members are solving the same problem.	Role clarity (leadership) Communication skills (call back of instructions) Management of personnel and resources (requesting help early) Decision-making skills (avoiding fixation errors)	Examined in aviation and military system. Improves performance and reduces errors. Emphasises the role of human factors in high-stress, high-risk environments. ACRM improved patient safety and issues related to resident supervision. Trainees develop an understanding of cognitive errors. Educating individuals about the limitations of human performance.	There is no universal CRM training programme. No definitive data link CRM to decreased incidents. Few studies use a control group. Low cost-effectiveness
TeamSTEPPS	Emphasise teamwork principles for improvement of quality and safety of the healthcare system. Focused outcomes include but not limited to improved communication, effective teamwork and reduced error rates.	Leadership, situation monitoring, mutual support, communication	International use; clinical outcome benefits (mortality); long-standing; rigorous teaching and support framework; conflict resolution taught as part of wider teamwork framework.	Lack of time, and the inability to hold an individual accountable. No definitive data link TeamSTEPPS to decreased incidents. Lack of control group
‘Two-challenge rule’ (advocacy-inquiry)	Assumption of collective responsibility and accountability on patient safety and outcome. Equal responsibilities across both superiors and trainees. Identification of possible risks before they turn into actual issues.	The team members voice their concern by advocating and asserting their statement at least twice if the initial assertion is disregarded. These two attempts might come from the same individual or two different team members. The first challenge should be in the form of a question. The second challenge should provide some support for their concern.	Defines roles and responsibilities of the individual with concerns and the individual with whom these are being raised. Requires no specialised language. Intended to depersonalise the conflict discussion. Based on aviation cockpit rules. It can be used to bring up certain information that must be addressed. It can be used to resolve conflict. It can be taught effectively, as it relies on a collaborative language of clinical knowledge and procedural skills.	Relies on both sender and receiver understanding and ‘buying into’ the concept. A limited number of studies were identified using this training programme. There was no attempt in studies to demonstrate that this educational intervention improved patient safety in an actual clinical environment.
Team behaviour manipulation	The ability to speak up is dependent on the team culture and/or behaviours and not just on the person speaking up.	Team culture and behaviours are manipulated in an attempt to facilitate conflict resolution.	Acknowledges the importance of the wider team and that the barrier to speaking up may be external to the speaker. Closer alignment to real world where culture and behaviours are more likely to influence actions than training programmes.	Relatively diffuse May be compromised by actions of single individuals. Studies can demonstrate that the manipulation works, but do not provide a mechanism for creating the change in the real world.

ACRM, anaesthesia crisis resource management; CRM, crisis resource management; TeamSTEPPS, Team Strategies and Tools to Enhance Performance and Patient Safety.

Excluded studies

Five studies^{10 19–22} (online supplementary appendix 3) were excluded at full text screening stage for the following reasons: (A) no intervention related to speaking up training (n=3); (B) focused on conflict resolution and speaking up in general medical practice (n=2).

No studies were excluded due to overall high risk of bias (online supplementary appendix 4).

Context and duration of intervention

Variation in study intervention duration was observed, ranging from part-day¹⁴ training to short 30–45 and 50 min workshops,^{15 16} and 1-hour session.¹³ Outcomes were generally assessed by subjective methods rather than objective methods. Interventions were delivered through various contextual settings—classroom, training sessions and simulation centres.

Intervention outcomes

Friedman et al ¹³ assessed if education training intervention enabled resident anaesthetists to effectively challenge a clear wrong decision-making by their superiors during crisis in practice. They used a 1-hour interactive lecture which outlined the approaches of CRM and conflict resolution tools in the intervention group and a 1-hour interactive lecture of CRM only in the control group. The performances were randomly assessed by two independent raters using the modified Advocacy-Inquiry Scale (mAIS). This includes five scoring levels with the addition of a sixth level when a trainee endeavours to take over management of the case.

Observed results from the short-teaching CRM intervention showed a significant improvement in the participant’s ability to challenge their superior’s wrong decision. The inter-rater reliability of the mAIS was excellent and mAIS was higher in the intervention group than in the control group. The intervention group repeated effective challenge skills.

The results support first and second levels of Kirkpatrick’s (table 3)⁸ theoretical framework. The course was positively viewed by participants and led to improved learning, which in turn led to behavioural changes. However, reliability might be affected by performance bias as the scenario was created to show very obvious basic clinical mistakes. The mAIS has been used several times by the same group with reasonable inter-rater reliability.

Table 3.

Kirkpatrick’s hierarchy model⁸

Level 1: Reaction	The degree to which participants find the training favourable, engaging and relevant to their jobs.
Level 2: Learning	The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training.
Level 3: Behaviour	The degree to which participants apply what they learnt during training when they are back on the job.
Level 4: Results	The degree to which targeted outcomes occur as a result of the training and the support and accountability package.

Sawyer et al ¹⁴ investigated the TeamSTEPPS training programme including: leadership, situation monitoring, mutual support and communication. They used a prospective preassessment and postassessment design, with teamwork skills as the main outcome variable. These outcomes were assessed via writing of scripted medication orders and performance errors. Assessments of group work skills were measured before and after the training course using the previously validated TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ).²³ It includes 22 specific teamwork behaviours broken into the five TeamSTEPPS core competencies. Significant improvements in the following factors were observed: team structure, situation monitoring, leadership and communication (p<0.001), following training programme completion. Furthermore, requests for clarification from nurses on scripted drug order errors doubled between pretraining and post-training, from 38% to 77%,¹⁶ reflecting an increase in awareness and individual participant confidence. However, there was a potential follow-up bias in this study, as evaluations of the main outcomes were performed by the same researchers who observed the simulation setting in real time and were colleagues of the participants. The study also relied on self-reporting of attitudes and confidence as primary outcomes (Kirkpatrick level 2), which was by definition subjective and often inaccurate; this may be viewed as a major limitation.

The two-challenge rule (conversation skills, advocacy and inquiry; a component of the TeamSTEPPS approach) has been investigated in two studies from the same group^{15 16} in two operating room simulation environments. Both studies sought to determine whether the conversational technique of the programme could be effective in enhancing speaking up in practice.

Pian-Smith et al ¹⁵ randomly presented two scenarios to trainees with a 30–45 min educational aspect in both cases, while Raemer et al ¹⁶ used a 50 min workshop on speaking up on interventional group and control group, with an ‘interactive didactic’ presentation of basic patient safety.

Contrasting results were observed. The first study¹⁵ reported an increased frequency in the use of the two-challenge rule advocacy-inquiry by participants after programme implementation (16% pre and 72% post). However, the second study¹⁶ observed no improved speaking up behaviours among the intervention compared with the control group (p=0.42). Similarly, Pian-Smith et al ¹⁵ observed that anaesthetists’ verbal challenges to nurses did not change (p=0.84). This could be a clinical practice anomaly, where junior doctors might not fully realise the important role that nursing skills can play during a critical event, including managing obtainable resources and providing clinical expertise.

Both studies describe deliberate processes to provide reliability of observations, with independent, blinded scoring by two investigators on video-recorded scenarios. Objective measures of reliability were not reported.

Their findings may be somewhat limited by the fact that the training intervention was immediately before the post-test of communication in a simulated environment. Therefore, the participants could be considered to be primed to ‘know the answer’ to the simulation scenario and that performance in that scenario may not be applicable in other situations.

An alternative approach of manipulating the social context in which conflict occurs has been investigated in two studies from the same group^{17 18} in simulated crises.

Sydor et al ¹⁷ randomised scenarios to a hierarchical or non-hierarchical team structure in a simulated combined ethical and life-threatening crisis (administration of blood to Jehovah’s Witness). The outcomes were trainee anaesthetists’ capability to challenge an unethical decision by their superiors. Friedman et al ¹⁸ randomised trainees to scenarios with either a strict/exclusive or open/inclusive communication dynamic to investigate the impact of interpersonal behaviour of consultant anaesthetists on trainee anaesthetists during emergency airway practice. Performances were measured by using the mAIS by independent raters.^{17 18} There was a good inter-rater reliability of mAIS scores in both studies.^{17 18}

The first study¹⁷ did not demonstrate a significant impact of the operating team hierarchy level on trainees’ ability to challenge their superiors. The checking and administrating of blood between both groups (hierarchical and non-hierarchical) were not significant. Similarly, Friedman et al ¹⁸ observed that the interpersonal behaviour of the consultant anaesthetist and the resulting speaking up did not significantly impact on trainees’ ability to challenge their superiors during airway crisis.

Although these studies^{17 18} have similar sample sizes to other studies included in this review, they did not demonstrate a positive result. The reason for these insignificant results might be due to a lack of appropriate and effective training in challenging authority. The trainees’ lack of training itself in this system could be described as hierarchical.

Discussion

This study provides a comprehensive review of the evidence of impact on a range of outcomes for conflict resolution strategies in healthcare systems. The included studies investigated three training programmes: CRM, TeamSTEPPS and the two-challenge rule (which is itself a part of TeamSTEPPS), and a social approach adjusting the behaviour of other team members. All studies included in this review were conducted in the USA and Canada, possibly because all three training programmes are relatively new and were originally designed for the aviation industry in the USA. Each of these training interventions is predicated on the assumption that failure of those further down the hierarchy to challenge those ‘higher up’ is an underlying problem. The tools and techniques of the training interventions do not seem to directly address the ability of those higher up to ‘receive’ challenge. A key underlying theme of these programmes is the enhancement of healthcare professionals’ frequency of speaking up, thus empowering trainees to assertively challenge their superiors in a positive and effective manner when patient safety is a concern.

Recent studies have indicated that authority differences between team members can contribute to speaking up failures.^{10 19} This may be particularly evident in interactions between consultants and their trainees, given that the consultant has a direct influence on trainees. A reason for the poor speaking up may be that it is psychologically difficult to challenge directives from an authority figure; in other words, it is hard to speak up to senior anaesthetists or surgeons.¹⁵ This is possibly because of differences in experience, which has been found to be a major factor affecting position in a hierarchy.¹⁹ Additionally, none of the included studies specially focus on speaking up about unprofessional behaviours. A recent study²⁴ reported that 75% of the respondents observed unprofessional behaviours, however, reported speaking up about unprofessional behaviours was uncommon.

Morrison’s model of employee speaking up supports this assumption. It illustrates that the effect of performance and perception of autonomy were considered to be under the individual’s control; these have been reported as influencing factors relating to speaking up in the healthcare system.²⁵ He found that hospital policy and interdisciplinary policymaking are the wider contextual factors that impact on speaking up among professionals. Interestingly, all the interventions had a predominant focus on the individual at the ‘lower’ end of speaking up; there appears to be very little emphasis on those at the ‘top’ listening down, or fostering an environment where this is possible or encouraged. The Canadian studies are a welcome addition to this work, investigating (although with negative results) the impact of the wider team. There is a suggestion that speaking up is important for people who work in an operating theatre, and is related to other similar situations in health settings, as it is one part of conflict resolution.²⁵

The CRM principles focus on interpersonal communication, leadership and decision-making in the workplace. CRM training has produced positive outcomes in different industries, such as healthcare, aviation and military.^{26 27} In healthcare, in particular, a previous study²⁷ showed that a CRM programme could effectively improve various CRM skills in a variety of clinical events. Beyond teaching skills, CRM also seeks to improve participants’ ability to debrief others or explain critical events, such as medical mistakes. In line with this, Gaba²⁸ adapted the CRM model used in aviation into the field of anaesthesia (anaesthesia crisis resource management, ACRM). The ACRM programme aims to educate anaesthetists in non-technical skills, particularly group work, by using simulation environments. Most of the outcome measures were long term, followed up again at a later stage (1 year later) to demonstrate that the training had a long-term impact. A number of researchers reported that such a training programme is highly effective in enhancing clinical functioning, and recommend that should be taken yearly.²⁹ However, our review, with its relatively narrow focus, did not find convincing evidence of benefit.

TeamSTEPPS, in contrast, was specially designed for healthcare system to help improve safety culture and teamwork based on CRM principles. Many healthcare centres worldwide have implemented the TeamSTEPPS programme.³⁰ They have overall concluded that TeamSTEPPS programmes are a positive step towards supporting safety culture and teamwork. Although there is some evidence to support the effectiveness of TeamSTEPPS, there is still much to learn about its precise impact on safety outcomes.³¹ The main outcome measurements in the included studies were only short term. Other studies of TeamSTEPPS support its wider efficacy in improving patient outcomes overall.^{32 33}

The two-challenge rule as a single intervention lacks evidence of its efficacy, therefore its true benefit remains unknown, though it is a component of TeamSTEPPS. A possible explanation for the lack of its effect in the reported studies could be that the training period was short (30–45 min and 50 min, respectively),^{15 16} while speaking up behaviours are deeply rooted and difficult to alter within this period. It may also be that the effect relies on a wider understanding and context of positive team behaviours, as delivered through the TeamSTEPPS programme. Raising awareness of the key factors that contribute towards the ‘hurdles’ and ‘enablers’ in speaking up behaviours may help team leaders and decision makers to develop the quality and safety of healthcare in their settings. However, when delivered in the context of the TeamSTEPPS, it appears to have the potential to be effective. These conclusions should not be interpreted as dismissing the evidence of benefit of TeamSTEPPS training per se (beyond the confines of conflict resolution).^{30 31}

The studies exploring the impact of other team members’ behaviour are worthy of note. They address a potentially more pragmatic and real-world approach. Training programmes (if they are effective at all) are resource intensive, and only directly impact those who attend. Changing the culture of how clinicians work has the potential for far greater reach and with less direct resource costs. Although the studies produced negative results, it is likely that the potential effect size of the intervention was too small to be seen.

Several studies have shown that trainees within a hierarchy are usually unwilling to challenge incorrect decisions made by their superiors,^{15 17 22} even when it is clear that the decision involves unethical practice or might harm patients. The well-known case of Elaine Bromiley, a patient who died from hypoxic brain damage after intubation failure by two consultants, could be viewed as a typical scenario. Although other team members had the requisite knowledge to correct the problem, they were unable to speak up to the senior members, which, in turn, directly influenced patient safety.³⁴ In a similar case, a patient had the wrong kidney extracted despite a trainee member of the teamwork recognising that the healthy kidney was being operated on.³⁵ These cases indicate that an unwillingness to challenge hierarchy in the operating theatre can be associated with adverse events that threaten patient safety.

Each group or researchers has used a different set of measures to define ‘success’ of their intervention. Although there is evidence of validity (in its widest sense) for all of these measures, none can be described as completely robust. Data are limited for all their clinimetric properties. Although the T-TAQ has been formally validated, that is in the context of team attitudes, not the ability of individuals (or teams) to resolve conflict per se.

Although there is evidence of validity (in its widest sense) for all of these measures, none can be described as completely robust. Detailed reliability and validity aspects of the included studies are beyond the scope of this paper. There are major limitations in the studies reported in this review. First is the limited definition of ‘success’ used in the studies—often short term, or surrogates for real patient impact. Second is that most of the studies used speaking up, and did not address other skills, such as negotiation skills, required for conflict resolution.

We used Kirkpatrick’s framework of educational outcomes to provide a differentiation between educational, behavioural and patient-level outcomes. While the model itself can be criticised,³⁶ we believe it is a widely understood concept that provides useful anchors.

An underlying theme of the above educational programmes is the enhancement of healthcare professionals’ ability to speak up and enable trainees to challenge their superiors in a positive and effective manner when patient safety is a concern. There is less focus on seniors’ ability to receive this challenge, even though this is identified as a factor enabling/inhibiting challenge. The published studies generally provide support for level 1 and level 2 outcomes in Kirkpatrick’s theoretical framework. There is relatively sparse evidence of longer term benefit, and studies suffer from a lack of control groups and objective, validated measures.