The Relationship Between Technical Errors and Decision Making Skills in the Junior Resident

Abstract

Objective

The purpose of this study is to co-evaluate resident technical errors and decision-making capabilities during placement of a subclavian central venous catheter (CVC). We hypothesize that there will be significant correlations between scenario based decision making skills, and technical proficiency in central line insertion. We also predict residents will have problems in anticipating common difficulties and generating solutions associated with line placement.

Design

Participants were asked to insert a subclavian central line on a simulator. After completion, residents were presented with a real life patient photograph depicting CVC placement and asked to anticipate difficulties and generate solutions. Error rates were analyzed using chi-square tests and a 5% expected error rate. Correlations were sought by comparing technical errors and scenario based decision making.

Setting

This study was carried out at seven tertiary care centers.

Participants

Study participants (N=46) consisted of largely first year research residents that could be followed longitudinally. Second year research and clinical residents were not excluded.

Results

Six checklist errors were committed more often than anticipated. Residents performed an average of 1.9 errors, significantly more than the 1 error, at most, per person expected (t(44)=3.82, p<.001). The most common error was performance of the procedure steps in the wrong order (28.5%, P<.001). Some of the residents (24%) had no errors, 30% committed one error, and 46 % committed more than one error. The number of technical errors committed negatively correlated with the total number of commonly identified difficulties and generated solutions (r(33)= −.429, p=.021, r(33)= −.383, p=.044 respectively).

Conclusions

Almost half of the surgical residents committed multiple errors while performing subclavian CVC placement. The correlation between technical errors and decision making skills suggests a critical need to train residents in both technique and error management.

ACGME Competencies

Medical Knowledge, Practice Based Learning and Improvement, Systems Based Practice

Introduction

Central venous catheter (CVC) placement is a common procedure in the intensive care unit (ICU) setting. In the United States, over five million catheters are placed on a yearly basis (1). Common indications for central line placement include administration of vasoactive medications, rapid resuscitation, total parenteral nutrition, and delivery of caustic medications (2). When considering the options for central venous access, placement of a subclavian CVC is often preferred due to its low infection rate (3).

Though considered a simple procedure, central line insertion is not without risk. It is estimated that up to 15% of patients who undergo central line insertion will be confronted with at least one complication, including infection, arterial puncture, pneumothorax, hemothorax, mediastinal hematoma, and vascular thrombosis (1, 3, 4, 5). Data shows that experience and proper training are crucial to minimize such complications (5, 6). At tertiary care centers, residents are often responsible for inserting CVCs (6). Because resident experience can be highly variable, education is incredibly important to help minimize complications (6, 7). In addition, confirmation of adequate resident skill is in critical need (8).

Resident education continues to improve with goals of decreasing patient morbidity. In central line training, simulation helps house staff develop baseline procedural skills prior to working on patients (8, 9). Other efforts at decreasing resident errors include video review of real time performance (10). While valuable in providing important technical assessment, these studies are limited by providing an incomplete assessment of CVC insertion. Recent studies have improved the process of assessment, recognizing the importance of decision making and encouraging the incorporation of broader assessment methods (11).

The purpose of this study is to improve upon current resident assessment methods and evaluate both resident technical errors and decision-making capabilities during placement of a subclavian CVC in a simulated environment. We predict that there will be significant correlations between scenario-based decision-making skills, and technical proficiency in central line insertion. In addition, we predict that residents will have some problems in anticipating common difficulties and generating solutions associated with line placement.

Materials and Methods

Setting and Participants

As a part of a longitudinal skills decay study, general surgery programs in the midwestern United States were sent inquiries for participation. Recruitment efforts were by phone or electronic correspondence to either program directors or resident education coordinators. Seven programs agreed to participate. Primary participant recruitment efforts focused on first year research residents, however, clinical residents and research residents in their second year who expressed interest in participation were not excluded. Data collection occurred at the respective training program’s location during the summer of 2014 and all participation was voluntary. This study was approved by the University of Wisconsin Hospital and Clinics (UWHC) Institutional Review Board.

Research Protocols

Participants first completed a demographic survey. Once completed, they were guided to the central line station. A research assistant read a standardized introduction to the central line task.

Subclavian Central Line Insertion

Participants were provided a clinical scenario of an elderly male in septic shock in need of a central line. Participants were asked to insert a subclavian central venous catheter on a central line simulator that has a realistic anatomical representation of the right upper torso of an adult male, including the subclavian, internal, and external jugular veins, and carotid and subclavian arteries. Arterial blood was represented with red liquid and venous blood by blue liquid. While there is an evolving standard for ultrasound guided internal jugular central lines, subclavian was chosen because the simulator used was not compatible with ultrasound. As such, both subclavian and internal jugular accesses were viewed as equally appropriate options (3).

The patient was presumed to have given consent, have no contraindication to the procedure, placed in Trendelenburg position, prepped and draped in the usual fashion, and already injected with local anesthetic. Participants were encouraged to verbalize any steps they would perform normally but could not complete due to the limitation of the simulated environment.

Participants were provided with a full central line insertion kit and a total of 15 minutes to complete the procedure portion of the task. Due to limitation of the simulator, a scalpel handle without a blade was provided to participants who were asked to still go through the motion of scalpel usage to represent the order in which it would have been used.

Subclavian Cognitive Scenarios

When the simulation was completed, participants were directed to the second portion of the station, where they were presented with a cognitive scenario. The scenario was a morbidly obese 36 year old female, post-operative day 2, in septic shock and in need of transfer to the intensive care unit for vasoactive therapy. Participants were provided with a picture of the patient’s upper torso (Figure 1). Participants were then asked to describe any potential difficulties they anticipated given the scenario and provide solutions to the difficulties when placing a subclavian central line.

Figure 1.

Patient picture presented to participants for cognitive scenario. Participants were asked to anticipate difficulties and generate solutions when considering insertion of a subclavian CVC. https://psnet.ahrq.gov/webmm/case/221, Accessed 03/16/2016.

Data Analysis

Study data were collected and managed using REDCap electronic data capture tools (12). REDCap is a secure, web-based application designed to support data capture for research studies. Data collection focused on a checklist of technical errors of varying morbidity (Appendix). The checklist used to track participant technical errors was adopted based on a literary review of the most commonly performed errors in subclavian central line insertion. The checklist was reviewed by an expert for final approval. Verbal responses to the cognitive scenarios were transcribed. Anticipated difficulties and solutions were grouped by similarity in theme. As an example, if participants listed multiple positioning maneuvers to optimize subclavian CVC placement as a solution, the responses were grouped as positioning maneuvers.

Data analyses were performed in SPSS 23 (13). Descriptive statistics and chi-square analyses were performed to evaluate technical error rates. Using a 5% error rate as the expected, chi-square analyses were performed on all errors. Pearson correlations were performed between the total number of errors on the central line and the number of commonly identified difficulties and solutions given in the central line scenario.

Results

Forty-six general surgery residents (53.2% female) performed this task. Participants were between their second and fourth post-graduate year. Forty-seven percent of residents were in their first research year, 36.2% were in their second research year, and 14.9% were in their second through fourth clinical years. Residents’ years of clinical experience varied: three participants completed one clinical year, 32 participants completed two clinical years, ten participants completed their third clinical year, and one completed their fourth clinical year.

Subclavian Central Line Insertion

Participants demonstrated poor needle angle more often than expected (26.7% of participants, ${\chi }_1^2=44.47$, p<.001). Participants also redirect the needle without adequate withdrawal more than expected (15.6% of participants ${\chi }_1^2=42.19$, p<.001). Less than two attempts to access the vein with the needle was coded as acceptable; however, more than 22% of participants took more than two attempts to access the vein with a maximum of seven attempts (M=2.04, SD= 1.77, ${\chi }_1^2=28.10$, p<.001). 13.6% of participants required more than one guidewire ( ${\chi }_1^2=6.91$, p<.009). A greater than expected number of participants forgot to use the scalpel (18.6%, ${\chi }_1^2=16.76$, p<.001). 13.9% of participants dropped the guidewire on the ground during the procedure ( ${\chi }_1^2=7.26$, p<.007). 11.6% fail to confirm placement of the catheter and 28.5% of the participants performed the steps of the central line procedure in the wrong order ( ${\chi }_1^2=3.98$, p<.046 and ${\chi }_1^2=47.82$, p<.001 respectively) (Figure 2).

Figure 2.

Percent of participants performing central line errors. Solid black line represents expected error rate, used for chi-square analysis. * represents errors committed in a statistically significant fashion ( p < 0.05). Yellow bars represent errors that were committed but did not achieve statistical significance.

Current assessment literature does not describe standard error expectation rates for technical performance of subclavian CVC placement. The American College of Surgeons/Association for Program Directors in Surgery Residents Skills (ACS/APDS) curriculum lists central line insertion a phase 1 skill (14). Based on the expectation set by this curriculum that a surgical resident should competently place a central line while in junior training, we used one error per participant as our expected error rate. Participants averaged a greater than expected 1.87 errors (SD=1.5, t(44)=3.82, p<.001). A minimum number of participants performed no errors (24%). 30% committed one error. 46 % committed more than one error. 2% committed a maximum of 6 errors (Figure 3).

Figure 3.

Percent of participants committing frequency of technical errors.

Subclavian Cognitive Scenarios

The three most common anticipated difficulties reported by participants for the cognitive scenario included difficulty associated with the patient’s obese body habitus (63%), difficulty in identifying land marks (48%), and dense soft tissue (35%). The three most common proposed solutions to overcome the anticipated difficulties included use of ultrasound (41%), placing an internal jugular central line instead of a subclavian (41%), and positioning maneuvers (24%).

The Relationship between Technical Errors and Cognitive Performance

The number of total errors performed during the subclavian central line procedure negatively correlated with the number of possible difficulties that residents could list while placing a central line (r(33)= −.419, p=.021). The number of total errors showed a significant negative correlation with the number of top solutions that residents could generate in response to anticipated difficulties in placing a subclavian central line (r(33) = −.383, p=.044). There was no correlation between number of completed clinical years, technical errors, and cognitive performance (p>.05).

Discussion

This study examines surgical residents’ technical skill and decision-making capacity while placing a subclavian venous catheter. We predicted there would be significant correlations between scenario-based decision-making skills and technical proficiency in central line insertion. In addition, we predicted that residents will have some problems in anticipating common difficulties and generating solutions associated with line placement. Our analyses indicate that most residents commit at least one cognitive or technical error while placing a subclavian venous catheter. The most common error committed by residents was performing the procedure in the wrong step order, performed by 28.5% of participants. Additionally, there were two significant negative correlations between the number of errors committed in line insertion and: (1) the number of anticipated difficulties, and (2) proposed solutions in difficult line placement scenarios. In other words, the participants who were more prepared to discuss difficulties and solutions in the cognitive scenario also committed a lower number of procedural errors during CVC placement. We believe that our data is a reflection of our participants’ clinical experience in central line insertion.

Junior Resident Technical Performance in Subclavian CVC

The cognitive scenario was meant to illustrate a difficult central line scenario. As the presenting patient was morbidly obese, we anticipated participants would list the patient’s obese habitus, increased soft tissue, and difficulty in identifying land marks as the top issues (15). The increased volume of soft tissue would obscure land marks and increase the difficulty of inserting the central line needle in the proper location and direction (15). Additionally, the larger body habitus would likely translate to a deeper needle insertion, which would leave participants familiar with CVC insertions on an average-sized individual unable to rely on muscle memory while initially inserting the needle. Participants’ top responses were in line with our expectations.

The majority of participants committed at least one error. With central line insertion listed as a Phase 1 basic/core skill by the resident skills curriculum (14), we presumed that the majority of residents would have no errors. This unexpected finding could be a disconnect between curricular expectation and current practices. With the introduction of peripherally inserted central catheter (PICC) insertion and Interventional Radiology teams, the option to consult other services for central venous access may be a more common option during real-life scenarios. This could translate into fewer opportunities for junior residents to practice central line insertion in early training, leading to a lack of mastery and committing more procedural errors.

One potential solution for the training gaps and limited exposure to central line insertion could be through simulation. Clinical practice in simulation has led to increased procedural performance and improved patient safety (8, 17). Practicing in simulation could allow for multiple opportunities to train in a protected environment multiple times without harming a patient. Moreover, simulation could also be used after competency is achieved to test difficult scenarios and maintain baseline skills. The ACS/APDS surgical skills curriculum already provides the blueprint curriculum and could be further built upon with cognitive scenarios and error checklists to offer a more comprehensive assessment (14). Training centers could utilize this further developed curriculum to identify gaps in training, ensure competency, and maintain skills for procedures that may not be as often performed.

Junior Resident Cognitive Performance in Subclavian CVC

There was a significant negative correlation between the number of errors performed on the subclavian central line insertion and the anticipated difficulties and proposed solutions with our cognitive scenarios. Participants with more errors on the procedural portion of the study have less anticipated difficulties and solutions with the cognitive scenarios. This is likely related to experience with central line insertions. Several studies show that more experienced residents and clinicians have higher success and fewer complications when inserting CVCs relative to those less experienced (6, 7). This may be because central line insertion opportunities allow for greater exposure to challenges, allowing for mentored guidance and troubleshooting. These exposures and guidance likely translates into improved performance and ability to troubleshoot difficult catheter scenarios independently.

Implications for Surgical Education

Transformation of surgical education and concerns for patient safety necessitates novel teaching and assessment methods in training programs. This will facilitate competency in a limited period of time for both bedside and operating room procedures (8). Examples of such methods in central line insertion include simulation practice and post-performance video review of technical aspects (8, 10). Recent studies indicate that assessment of cognitive skills in conjunction with technical skills is important for comprehensive assessment of procedural performance (11). Our study is one of the first published that attempts to simultaneously understand cognitive and technical performance of the subclavian central line insertion in the resident population. Exploring performance by this method appreciates how these skills vary and relate by differences in experience level.

Our findings should be further expanded upon in future works for improvement of surgical education. With our primary purpose being to co-evaluate technical and decision making skills, we wanted to show that simple methods could be developed and utilized in order to successfully achieve a more comprehensive assessment of surgical performance. We suggest that current, classic technical rating scales include evaluation of decision-making skills and other complex errors. With continued evolution in clinical assessment, it is possible to develop an objective method of technical and cognitive assessment—a goal that is strongly sought after in the field of surgery (11, 18)

While achieving the objectives of the study, some limitations were apparent. Our scoring checklist assesses technical aspects of the subclavian central line procedure; however, it does not appreciate the magnitude that an error could contribute to patient morbidity. Exploring the weight of committed errors would allow us to better appreciate the relationship between high magnitude errors and post graduate year, if applicable. The lack of statistical correlation between clinical experience with technical or cognitive skills is surprising. Intuitively, the expectation would be that clinical years of experience would directly relate to improved performance of central line insertion. Yet, we believe that a possible reason for the lack of statistical significance could be the sample size and breadth of sample. While we primarily focused on recruiting junior level residents, future work should evaluate resident performance with a larger, more diverse population.

Conclusion

This study suggests that junior residents current have lower than expected performance in Subclavian CVC insertion. Our results indicate current trainees are not gaining the necessary experience needed to achieve competency. There is a need for increased opportunities that develop subclavian CVC insertion mastery. Improvement in surgical education programs by use of simulation-based assessments that focus on technical and decision making skills may help to increase successful performance.

Appendix