ROLE CONFUSION AND SELF ASSESSMENT IN INTERPROFESSIONAL TRAUMA TEAMS

Abstract

Background

Trauma care requires coordinating an interprofessional team, with formative feedback on teamwork skills. We hypothesized nurses and surgeons have different perceptions regarding roles during resuscitation; that nurses’ teamwork self-assessment differs from experts’, and that video debriefing might improve accuracy of self-assessment.

Methods

Trauma nurses and surgeons were surveyed regarding resuscitation responsibilities. Subsequently, nurses joined interprofessional teams in simulated trauma resuscitations. Following each resuscitation, nurses and teamwork experts independently scored teamwork (T-NOTECHS). After video debriefing, nurses repeated T-NOTECHS self-assessment.

Results

Nurses and surgeons assumed significantly more responsibility by their own profession for 71% of resuscitation tasks. Nurses’ overall T-NOTECHS ratings were slightly higher than experts’. This was evident in all T-NOTECHS subdomains except “leadership,” but despite statistical significance the difference was small and clinically irrelevant. Video debriefing did not improve the accuracy of self-assessment.

Conclusions

Nurses and physicians demonstrated discordant perceptions of responsibilities. Nurses’ self-assessment of teamwork was statistically, but not clinically significantly, higher than experts’ in all domains except physician leadership.

Introduction

Early care of the injured patient is a complex, time-critical endeavor, requiring the coordination of an interprofessional trauma team. Preventable deaths occur even in mature trauma centers, with a third of errors occurring during the initial evaluation and resuscitation in the emergency department. (1, 2) Lapses in teamwork (non-technical skills) are postulated to be a major source of error, and provide a target for training and performance improvement efforts. (3–5) Excellent teamwork mandates rapid and appropriate role assignment and task allocation (6), which can be difficult in the “extreme action” venue of trauma, where teams are assembled ad hoc, comprised of interdisciplinary team members of varying titles, training and skills. The evolving roles of advanced clinical practice nurses, flattening of the traditional physician-leader hierarchical culture, and a dynamic delegation style of leadership (7) may contribute to confusion regarding relative physician versus nurse responsibilities during resuscitation.

In addition to a shared mental model regarding team members and roles, optimizing trauma teams requires ongoing, accurate assessment and feedback about teamwork process. We previously developed a rating tool, T-NOTECHS, for evaluating teamwork and providing formative feedback to trauma teams. (8) We found that T-NOTECHS ratings were more accurate and concordant following video review, compared to direct observation and immediate scoring, in simulated trauma resuscitations. However, videotaping and debriefing of actual trauma resuscitations can be costly, time consuming and may increase malpractice liability exposure. (9) Thus, the standard practice in many institutions, including ours, is to perform immediate physician-led debriefing of critical trauma cases. This practice may be limited by the reported inaccuracy of physician self-assessment of non-technical skills. (10, 11) Nurses have been pioneers in the field of interprofessional communication (12), and ostensibly might be more capable than physicians in assessing interprofessional teamwork. However, more data is needed regarding nurses’ accuracy in self-assessment and the relative benefit of video review versus immediate post-resuscitation rating and assessment.

We propose that nurses and surgeons on modern interdisciplinary trauma teams may have discordant views regarding their roles during trauma resuscitation. We further hypothesized that nurses’ self-assessment of teamwork may differ from expert assessment, and that video debriefing might improve the accuracy of self-assessment.

Methods

This study was conducted at a Level II trauma center, where ad hoc trauma teams are comprised of emergency medicine and trauma attending physicians, medical students, residents, physician’s assistants, advanced practice nurses and, for critically injured patients, an anesthesiologist, surgical critical care fellow and respiratory therapists. Emergency department trauma nurses who were enrolled in a trauma refresher curriculum and attending trauma surgeons gave informed consent to participate in this Institutional Research Review Committee approved study.

Nurses were surveyed regarding their background and prior team training. Trauma surgeons and nurses were then independently surveyed regarding their baseline perception of the relative responsibilities of nurses versus physicians during trauma resuscitation. Survey items included 17 typical trauma resuscitation tasks. Practitioners completed a 7-point Likert-type scale, scoring each task as the responsibility of a trauma nurse (score= 1), physician (score= 7) or an equally shared responsibility (score= 4).

Following the survey, nurses participated in 90 minutes of web-based didactic and live lecture, reviewing teamwork principles and non-technical skills assessment using T-NOTECHS (8). T-NOTECHS evolved initially from a non-technical skills (NOTECHS) scale developed for aviation, and is based upon five behavioral domains: Leadership, Cooperation and Resource Management, Communication and Interaction, Assessment and Decision Making, and Situation Awareness/Coping with Stress, illustrated by 47 behavioral exemplars. Domains were rated on a scale of 1–5, where 1 = no demonstration of the skill, and 5 = flawless demonstration of the skill. Subsequently during scheduled half-day sessions, groups of 3–4 nurses joined multidisciplinary teams including confederates (surgeon, emergency physician, respiratory therapist, resident), and completed four standardized, ten-minute, video recorded human patient simulator-based (SimMan, Laerdal) trauma resuscitation scenarios. Scenarios were presented in random order, and depicted blunt trauma and burn cases with critical pathophysiology and complex psychosocial, communication and resource management issues (Table 1).

Table 1.

Summary of Scenario Design with key elements for debriefing

Scenario Content	Teamwork Probe	Key T-NOTECHS Teamwork Domains	Observable Teamwork Behaviors	Clinical Care Concepts
Football Injury: Minor with ruptured spleen	Shoulder pads on patient Parents’ religion prohibits transfusion	Assessment and Decision Making Situational Awareness and Coping with Stress Communication Leadership	Primary survey completed after pads removed Transfusion decision communicated to members	Massive transfusion protocol Hemorrhagic shock/upgrading level of activation Consent conflict for minors
Motor Vehicle Crash (MVC): Intoxicated celebrity with facial trauma	Distracting “gawker” enters trauma room. Key team member called away/Handoff required Uncooperative patient	Situational Awareness and Coping with Stress Leadership Cooperation and Resource Management	Delegation of roles and responsibilities Explicit airway role handoff Management of distractions	Head trauma Alcohol confounder Multi-casualty incident
Pedestrian vs MV: 20 week pregnant woman, CPR on arrival	Poor CPR technique by confederate	Communication and interaction Leadership Assessment and Decision Making	CPR quality identified Communication of clinical priorities	Trauma in pregnancy Management of pulseless patient
Burn: Extensive burn in geriatric patient	Conflict among confederates regarding intubation/advance directives Telephone communication with family	Leadership Communication and interaction	Discussion of limits to care within team Communication of family member status to leader	Limits to care in nonsurvivable injury Advanced directives

Immediately following each scenario, teamwork was independently and confidentially assessed from memory (without video review) by nurses (“self-assessment”), and by two physician trauma teamwork “experts” (S.S. and B.B.) using T-NOTECHS. Experts were board-certified in Critical Care, each with >25 years of clinical trauma experience and >5 years in teamwork training, familiar with the use of T-NOTECHS and demonstrating satisfactory (Intraclass Correlation Coefficient = 0.71) concordance in T-NOTECHS rating of videotaped trauma resuscitations. After initial teamwork scoring, nurses participated in a 30-minute structured, facilitated video debriefing emphasizing T-NOTECHS domains and aligned with evidence-based recommendations for effective debriefing. (13) Following structured debriefing for each scenario, nurses independently repeated T-NOTECHS self-assessment teamwork scoring. Nurses were surveyed before and after completion of all the simulation scenarios regarding their confidence using T-NOTECHS and their perceptions regarding the ability of T-NOTECHS to accurately reflect essential elements of teamwork during trauma resuscitation.

Statistical analysis

Nurses’ baseline characteristics were described by mean and standard deviation, and frequency and percentage. Wilcoxon nonparametric test or two sample t test was used to analyze differences between surgeon and nurse perceptions of responsibilities for each resuscitation task. For each teamwork domain (range 1–5) and overall score (range 5–25), paired t tests were used to compare nurses’ average T-NOTECHS ratings for either pre-debriefing or post-debriefing to experts’ average ratings by scenario and team (not shown). In addition, we conducted mixed effect models to account for the repeated random subject effects on the difference between nurses’ ratings and average experts’ ratings, adjusting for scenario type and team. Similarly, to evaluate the impact of video debriefing, we used mixed effects models on the difference in average nurses’ pre- and post-debriefing ratings, adjusting for scenario type and team.

To show improvement in perceptions regarding utility and ease of using T-NOTECHS, two sample t-tests were used. P < 0.05 was considered statistically significant and all the analyses were conducted by a biostatistician in SAS 9.4.

Results

Nine trauma surgeons and 42 nurses (demographics in Table 2) completed the baseline survey. The cohort represents 90% of the center’s trauma surgeons and approximately half of our trauma nursing workforce. Nurses and surgeons differed significantly in their perception of the responsibility for 12 of the 17 (71%) resuscitation tasks. Both groups assigned significantly more responsibility to their respective own profession (Table 3). Surgeons and nurses agreed that surgeons are primarily responsible for determining patient code status, i.e. consideration of “do not resuscitate” orders for patients of advanced age, debilitation or poor functional status at baseline. Surgeons and nurses agreed on nurses’ responsibility for reconciling pre-hospital medications and communicating stable patient condition to the receiving ward. They further agreed that toxicology testing, and providing anxiolytics and analgesia were equally shared roles. However, surgeons and nurses each felt their own profession had primary responsibility for giving prophylactic medications, assuring gastric tube placement, pregnancy testing and hand-off communication to the intensive care unit (ICU).

Table 2.

Nurses’ Background

Variable	N (%)
Gender
Female	17 (40%)
Male	25 (60%)
Age (yrs.), mean ± SD	39.2 ± 10.2
Race/Ethnicity
Caucasian	17 (40%)
Asian	9 (21%)
Filipino	7 (17%)
Hawaiian/Other Pacific Islander	6 (14%)
Mixed	3 (7%)
Prior formal Team Training
Yes	22 (54%)
No	19 (46%)
Prior formal Team Training at our institution using T-NOTECHS
Yes	9 (21%)
No	33 (79%)
Trauma Team experience (yrs.), mean ± SD	6.7 ± 5.9
Clinical experience (yrs.), mean ± SD	10.3 ± 7.3

Table 3.

Surgeons’ versus nurses’ perceptions of responsibility for tasks. 1= Nurse responsible, 7= Physician responsible, 4=equally shared. Mean ± SD

TASK	Surgeon rating (N = 9)	RN rating (N = 42)	p-value
Primary survey (airway, breathing, circulation, neurologic)	6.7 ± 1.0	5.2 ± 1.2	<.001
Secondary survey (head-to-toe exam)	6.4 ± 0.9	4.9 ± 1.0	<.001
Review of lab results	6.1 ± 0.6	4.4 ± 0.8	<.001
Communication with consultants	5.9 ± 1.1	4.5 ± 1.0	<.001
Pregnancy testing	5.0 ± 1.5	2.7 ± 1.5	<.001
Assuring gastric tube is placed in an intubated patient	4.8 ± 1.6	3.0 ± 1.4	.001
Ensuring administration of prophylactic medications (anti-seizure, antibiotic, tetanus)	5.3 ± 1.2	3.6 ± 1.5	.002
Communication of patient status to receiving ICU (critical patient)	4.8 ± 1.6	3.5 ± 1.3	.008
Cleansing and dressing superficial wounds	3.8 ± 1.4	2.5 ± 1.4	.016
Communication with family	5.4 ± 1.0	4.6 ± 1.0	.026
Teaching trainees	5.3 ± 1.2	4.5 ± 1.0	.033
Maintaining normothermia	4.2 ± 1.1	3.2 ± 1.3	.034
Toxicology testing	4.8 ± 1.4	3.8 ± 1.5	.075
Determining code status for patients with poor functional status pre-injury	6.6 ± 1.0	5.7 ± 1.4	.080
Analgesia and anxiolytics	4.3 ± 0.9	4.0 ± 1.5	.467
Communication of patient status to receiving ward (stable patient)	3.0 ± 1.8	2.7 ± 1.3	.539
Recording pre-hospital medication and dosage	2.4 ± 1.4	2.4 ± 1.3	.936

30 nurses went on to complete all four resuscitation scenarios with video debriefing. Of this group, 52% had undergone prior formal team training, 20% at our institution using T-NOTECHS. Nurses’ overall T-NOTECHS ratings exhibited a small but statistically significant difference from expert ratings (Table 4). This difference was also evident in all domains except in the domain of Leadership. Video debriefing did not appear to have a significant impact upon nurses’ concordance with experts’ ratings. When evaluating differences by behavioral domain, nurses consistently rated all domains other than “leadership” higher than experts. Analysis of the subgroup of nurses who had undergone simulation-based T-NOTECHS training with video debriefing five years ago, revealed mean T-NOTECHS ratings closer to expert ratings, with a significant difference only in the domain of “Communication” (data not shown).

Table 4.

Comparison of mean (± SD) T-NOTECHS ratings between experts and nurses (pre- and post-debriefing). Mean (± SD) adjusted for team and scenario type.

T-NOTECHS DOMAIN	Expert	RN Pre Debriefing	RN Post Debriefing	RN Pre vs. Expert, p-value	RN Post vs. Expert, p-value	RN Pre vs. Post, p-value
Leadership	3.6 ± 0.5	3.6 ± 0.9	3.5 ± 0.7	0.74	0.41	0.70
Cooperation and Resource Management	3.4 ± 0.6	3.7 ± 0.7	3.7 ± 0.6	0.01	<0.01	0.55
Communication and Interaction	3.1 ± 0.6	3.5 ± 0.7	3.5 ± 0.6	<0.01	<0.01	0.46
Assessment and Decision Making	3.1 ± 0.7	3.4 ± 0.9	3.2 ± 0.8	0.04	0.15	0.08
Situation Awareness/Coping with Stress	3.3 ± 0.6	3.5 ± 0.8	3.4 ± 0.8	0.12	0.02	0.80
TOTAL	16.3 ± 2.3	17.6 ± 3.3	17.2 ± 2.6	0.01	<0.01	0.44

When assessing the perceived utility of T-NOTECHS, 73% of nurses initially “agreed or strongly agreed” that they were comfortable using T-NOTECHS. This improved to 87% after video debriefing (p < .05). 77% of nurses initially agreed that T-NOTECHS captured important elements of teamwork, this improved to 93% after debriefing.

Discussion

Modern trauma teams in academic centers have an increasingly complex composition. The expansion of the “team care” paradigm is driven in part by physician workforce shortages and resident work hour restrictions. Prior studies have shown that nurses may lack knowledge of other professionals’ background, training and skills. (14, 15). The uncertainty created by this knowledge gap may be exacerbated by the critical nature of trauma resuscitation, and may precipitate role confusion and a perceived need to retain more control in direct patient care. Our data confirmed our suspicion that trauma physicians and nurses have discordant perceptions of their roles during resuscitation. Specifically, a number of tasks traditionally under the purview of the physician (e.g. reviewing lab results and giving prophylactic medications) were considered by the nurses to be nursing responsibilities. Overlapping perceptions of domain and task may result in a beneficial effect by reducing errors of omission. However, the redundancy could also lead to deleterious effects such as poor time management during resuscitation, suboptimal resource utilization, and increased cost or patient harm due to duplication of tests or medications. “Followership” is emerging as an important component of teamwork training, dictating that team members build their competence and focus their efforts for maximum impact. This key element of teamwork can emerge only if team members have a shared mental model of team roles and responsibilities. (16) This concept of followership, in light of our data showing discordant team member perceptions, may inform teamwork training efforts toward increasing knowledge of others’ backgrounds and skills, and defining and reinforcing role assignments and task allocation.

Nurses play an essential role not only in the delivery of clinical trauma patient care but also in the performance improvement and quality assurance aspect of trauma systems. The enhanced training that nurses receive in communication may theoretically render them more accurate in assessing these nontechnical skills in interprofessional teams. However, our data suggest that nurses, similar to physicians, exhibit limitations in self-assessment, particularly of communication and coordination. Nurses’ assessment of the “leadership” demonstrated by physician confederates in the scenarios more closely approximated the experts’ ratings, and were uniquely lower than experts’ ratings only in this domain. This may illustrate the difficulty of criticizing one’s self and colleagues in the same professional domain, and is complementary to findings in our previous study, where physicians’ self-assessment of their own leadership was higher than experts’, and correlated poorly with objective performance metrics. (8) Though statistically significant, the difference between the nurses’ and experts’ ratings were small and arguably not clinically significant. However, we hope that our results may inform the design of future teamwork training endeavors and research that is better powered to explore this difference.

Institutional culture varies considerably and may affect perceptions of individual practitioners and their concepts of teamwork and roles, as emphasized in research regarding safety culture in healthcare organizations. (17) Our institutional culture during the time of this study was not formally assessed, though likely influenced our results, and thus limits the generalizability of our findings to other institutions. Future studies in this realm should optimally be multi-institutional, and should characterize organizational culture to better explain discordance and determine the particular effect of teamwork training interventions.

Debriefing is considered essential for high-quality simulation-based training, (13) and debriefing of actual critical events, such as trauma resuscitations, may have a positive formative effect on teamwork. (18) Traditionally, debriefings were done in real-time, and often immediately following resuscitation. (19) Efforts to optimize efficiency of team training and debriefing have employed self-debriefing or facilitator-free methods, with or without video review. The quality and lasting impact of trauma teamwork focused debriefing may be impaired by reliance upon inaccurate team self-assessment rather than extrinsic expert assessment, even when using a guide such as T-NOTECHS. Other investigators have shown a similar lack of reliability of nontechnical skills self-assessment in the operating room. (10, 20) Video review has been frequently used as a tool for training individuals in assessing teamwork (21), and video review of actual clinical trauma resuscitations has been used as a performance improvement tool in trauma centers (22). Furthermore, the technique of video debriefing has been adopted widely in teamwork training curricula, and with demonstrated process improvement in simulated resuscitations that has translated into clinical process improvement. (6, 23, 24) Thus, we postulated that the technique of structured video debriefing would enhance team members’ skills in self-assessment of teamwork.

Our study did not demonstrate a significant effect of video debriefing on teamwork assessment. Even after video review, a small but significant discord between the nurses and experts persisted in overall T-NOTECHS scored and in three of the five teamwork domains. Moreover, nurses’ pre- and post-debriefing T-NOTECHS ratings were not significantly different, although our ability to discern a difference was handicapped by pooling of post-debriefing ratings (via audience response system) which precluded individual comparison. The effect of video review may have been dampened by having the nurses rerate the teamwork after only a brief period of time (30 minutes) and in continued close proximity to the resuscitation. It is possible that delayed video review might allow a fresher assessment of teamwork that is less influenced by prior ratings or emotions during the resuscitation. The nurses showed greater accuracy in assessing physician (confederate) leadership than other domains. This, coupled with prior work demonstrating the inaccuracy of physicians’ self-assessment, would support the active engagement of nurses in interprofessional trauma team self-assessment.

Our study was limited by the relatively small number of participants from a single institution; all of the nurses were familiar with each other and this may have influenced their ability to objectively rate teamwork. The short time for retraining and familiarization with the T-NOTECHS tool may also have hampered accuracy. The reliability of T-NOTECHS in this setting was not as robust as demonstrated in post-hoc video review. The inter-rater reliability between the two experts was calculated with weighted kappa and intraclass correlation coefficients, and this was relatively low (≤0.25). This limits the generalization of T-NOTECHS in real-time rating of resuscitation and suggests the need for increased number and better calibration of raters in this realm and is a focus for future research.

Conclusion

Nurses and physicians demonstrated discordant perceptions of their relative responsibilities for trauma resuscitation tasks. Each group perceived a significantly greater sense of ownership for 12 out of 17 standard resuscitation tasks.

Nursing team members’ self-assessment of teamwork was more positive than experts’ assessment overall and in most domains. The exception was in their assessment of leadership demonstrated by the physicians (confederates) in the scenarios. Structured immediate video debriefing in this setting appeared to have no significant impact on teamwork ratings or concordance with experts.

This study corroborates others in demonstrating gaps in interprofessional knowledge and shared mental model, as well as the potential inaccuracy of self-assessment. It is unique in our finding of nurses’ greater accuracy in interprofessional assessment of physicians. Future research is needed on techniques to improve interprofessional knowledge and concordance of team roles, and the use of video debriefing, perhaps in a delayed fashion, in trauma resuscitation. We plan to continue research on the validity and reliability of T-NOTECHS in real-time and actual clinical settings, and expansion of our local experience to multi-institutional collaborative investigations.