Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists

Abstract

Background and objectives

Concerns have been raised about nephrology fellows’ skills in inserting temporary hemodialysis catheters. Less is known about temporary hemodialysis catheter insertion skills of attending nephrologists supervising these procedures. The aim of this study was to compare baseline temporary hemodialysis catheter insertion skills of attending nephrologists with the skills of nephrology fellows before and after a simulation-based mastery learning (SBML) intervention.

Design, setting, participants, & measurements

This pre- post-intervention study with a pretest-only comparison group was conducted at the University of Toronto in September of 2014. Participants were nephrology fellows and attending nephrologists from three university-affiliated academic hospitals who underwent baseline assessment of internal jugular temporary hemodialysis catheter insertion skills using a central venous catheter simulator. Fellows subsequently completed an SBML intervention, including deliberate practice with the central venous catheter simulator. Fellows were expected to meet or exceed a minimum passing score at post-test. Fellows who did not meet the minimum passing score completed additional deliberate practice. Attending nephrologist and fellow baseline performance on the temporary hemodialysis catheter skills assessment was compared. Fellows’ pre- and post-test temporary hemodialysis catheter insertion performance was compared to assess the effectiveness of SBML. The skills assessment was scored using a previously published 28-item checklist. The minimum passing score was set at 79% of checklist items correct.

Results

In total, 19 attending nephrologists and 20 nephrology fellows participated in the study. Mean attending nephrologist checklist scores (46.1%; SD=29.5%) were similar to baseline scores of fellows (41.1% items correct; SD=21.4%; P=0.55). Only two of 19 attending nephrologists (11%) met the minimum passing score at baseline. After SBML, fellows’ mean post-test score improved to 91.3% (SD=6.9%; P<0.001). Median time between pre- and post-test was 24 hours.

Conclusions

Attending nephrologists’ baseline temporary hemodialysis catheter insertion skills were highly variable and similar to nephrology fellows’ skills, with only a small minority able to competently insert a temporary hemodialysis catheter. SBML was extremely effective for training fellows and should be considered for attending nephrologists who supervise temporary hemodialysis catheter insertions.

Introduction

Temporary hemodialysis catheter (THDC) insertion is considered to be a core competency of nephrology fellowship training (1,2). Improper THDC insertion has the potential to cause life-threatening complications, including pneumothorax, carotid puncture, bleeding, and central line–associated bloodstream infection (3). However, surveys of graduating nephrology fellows in the United States and Canada indicate that up to one third do not feel comfortable inserting catheters (4,5). This perception is not unfounded, because studies in which THDC insertion skills have been objectively tested show that <20% of traditionally trained nephrology fellows can perform the procedure competently (6,7).

Simulation-based mastery learning (SBML) is superior to the traditional see one, do one, teach one method to develop THDC insertion skills among nephrology fellows (6,7). SBML is an extreme form of competence-based learning that requires learners to complete a structured curriculum, including pretesting, deliberate skills practice with feedback, and post-testing (8,9). Learners practice until they meet or exceed a predetermined minimum passing score (MPS) (8,9). Studies using SBML for central venous catheter insertion show that skills obtained in the simulated environment transfer to actual clinical care, resulting in lower complication rates, including reduced central line–associated bloodstream infections (10–13).

Prior research has analyzed the skills and confidence of nephrology fellows regarding THDC insertion and the benefit derived from SBML (6,7,14). However, little is known about the actual competence of supervising attending nephrologists to perform the procedure themselves. We hypothesized that, because of the nature of their original training, decay of skills over time, and lack of familiarity with advances in practice (15), THDC insertion skills of attending nephrologists would be highly variable. Therefore, this study had two aims: first, to assess baseline THDC insertion skills among attending nephrologists, and second, to compare the skills of attending nephrologists with those of nephrology fellows before and after an SBML intervention.

Materials and Methods

Study Design

We performed a pre- post-intervention study with a pretest-only comparison group over a 1-week period in September of 2014. We compared attending nephrologists’ baseline internal jugular (IJ) THDC insertion skills with skills of nephrology fellows before and after an SBML intervention. We performed this study at the University of Toronto, Mount Sinai Hospital, a quaternary care academic medical center in Toronto. All attending nephrologists at three University of Toronto academic affiliates were eligible to participate in the baseline assessment. All adult nephrology fellows (as an initiative of the University Health Network Division of Nephrology) were required to participate in THDC insertion SBML as part of their training. The University of Toronto Ethics Board and the Northwestern University Institutional Review Board approved this study, and all participants provided informed consent before participating.

Procedure

Attending nephrologists completed a baseline IJ THDC insertion simulation. Skills were assessed using a previously published 28-item skills checklist and a central venous catheter insertion simulator (6,7). Each attending nephrologist was assessed individually and asked to perform the THDC insertion as if it were an actual patient. During the simulation, an instructor read a standardized case of a patient requiring urgent hemodialysis. Participants had access to all necessary THDC insertion supplies as well as sterile gowns, masks, gloves, drapes, and an ultrasound device.

Nephrology fellows completed a previously described THDC insertion SBML intervention (6). Fellows performed a baseline skills assessment (pretest) of IJ THDC insertion skills in the same manner as attending nephrologists. Fellows then reviewed a lecture and video demonstration of proper THDC IJ insertion using real-time ultrasound guidance. Fellows then participated in 1 hour of deliberate practice with individualized feedback and guidance from faculty with expertise in THDC insertion. Before the intervention, faculty (R.F.M., E.C., and A.Z.) completed THDC SBML instructor training led by the instructor who developed the curriculum (J.H.B.). This training was conducted to ensure uniform grading of pre- and post-tests.

Subsequently, fellows underwent a skills post-test using the checklist and simulator. Post-tests were performed 1–5 days after training. Each fellow was required to meet or exceed the MPS set previously by a multidiscipilinary expert panel at 79% of checklist items correct (6). Fellows who did not meet or exceed the MPS initially completed additional deliberate practice until the MPS was reached.

All attending and fellow testing sessions were video recorded to assess for inter-rater reliability.

Measurement

All pre- and post-test assessments were recorded using the same 28-item checklist. Assessments were performed using Simulab’s CentraLineMan (16). This simulator is anatomically correct and features ultrasound compatible tissue, IJ and subclavian veins, and pulsatile carotid and subclavain arteries. The model can accommodate a full THDC insertion, including use of a dilator. Artificial blood can be aspirated, and saline/medications can be injected.

Participants provided demographic information, including age, sex, years of training and/or clinical practice, number of IJ THDCs inserted (experience), and self-confidence in performing the procedure. Self-confidence was reported using a scale of 0–100 (0, not confident; 100, extremely confident). After completing the SBML intervention, fellows were surveyed about their experience using a previously published survey instrument (6).

A 30% random sample of video-recorded assessments was reviewed by a second observer to assess inter-rater reliability. This observer (J.J.P.) has expertise in THDC insertion and experience teaching the SBML curriculum. The second observer was blinded to original test score, pre- or post-test status, and whether the study participant was a fellow or attending nephrologist.

Primary outcome measures were comparisons between fellows’ and attending nephrologists’ pretest (baseline) performance on the skills assessment and fellows’ pre- and post-test performances before and after the SBML curriculum. Secondary outcome measures were correlations between pretest checklist performance and demographic factors, THDC insertion experience, and self-confidence. We also report results from the fellows’ course evaluation questionnaire.

Statistical Analyses

We compared attending nephrologists’ pretest scores with fellows’ pretest scores using the independent t test. We evaluated differences between fellows’ pre- and post-test scores using the paired samples Wilcoxon signed rank test. We calculated Cohen’s κ to assess for inter-rater reliability between faculty graders and the blinded reviewer. We calculated Pearson’s correlation coefficients to assess relationships between demographic factors, prior IJ THDC insertion experience, and self-confidence with pretest scores. We performed all statistical analyses using IBM SPSS Statistics, version 22 (IBM Corp., Armonk, NY).

Results

In total, 19 of 36 (53%) attending nephrologists at the three hospitals participated in the study. All 20 nephrology fellows at the University of Toronto completed the entire protocol, including pre- and post-testing. Baseline data, including demographics, prior clinical experience, and procedural self-confidence, are reported in Table 1.

Table 1.

Demographics of study participants (n=39)

Characteristic	Fellows (n=20)	Attending Nephrologists (n=19)
Mean age, yr (SD)	35 (6)	44 (13)
Sex (% men)	17 (85)	9 (47)
Median years of fellowship training or as an attending nephrologist (IQR)	2 (1–5)	9 (515)
Median no. of IJ THDCs inserted in clinical practice (IQR)	21 (8–51)	12 (9–48)
Median IJ THDC insertion self-confidence (IQR)a	15 (5–448)	20 (10–50)

IQR, interquartile range; IJ, internal jugular; THDC, temporary hemodialysis catheter.

^aScale: 0, not at all confident; 100, extremely confident.

Attending nephrologists had a mean baseline checklist score of 46.1% (SD=29.5%). This was similar to the mean fellows’ baseline checklist score (41.1% items correct; SD=21.4%; P=0.55). Only two of 19 (11%) attending nephrologists and zero of 20 fellows met or exceeded the MPS at baseline (Figure 1).

Figure 1.

Attending nephrologists’ baseline temporary hemodialysis insertion skills compared with nephrology fellows’ performance before (pretest) and after (post-test) simulation training. MPS, minimum passing score.

After completing SBML, the fellows’ mean post-test score improved to 91.3% compared with baseline (SD=6.9%; P<0.001); 18 of 20 (90%) fellows met the MPS at initial post-test. After <1 hour of additional practice, the two remaining fellows exceeded the MPS.

Inter-rater reliability for checklist items was high (κ=0.71).

THDC insertion self-confidence correlated moderately with pretest performance (r=0.36; P=0.03). Additionally, younger age and fewer years of experience as an attending nephrologist correlated with better pretest performance (r=−0.45; P=0.004 and r=−69; P=0.001, respectively). Sex, years as a nephrology fellow, and number of THDCs inserted in clinical practice did not correlate with pretest performance.

Overall, fellows rated the SBML curriculum highly (Table 2). All fellows agreed or strongly agreed that simulation training should be a required component of nephrology fellowship.

Table 2.

Responses to a postsimulation–based mastery learning intervention course evaluation questionnaire (n=20)

Item	Strongly Disagree	Disagree	Uncertain	Agree	Strongly Agree
Practice with the central line model boosts my skill to perform this procedure	0	0	0	3	17
Educational sessions using medical simulation are fun	0	0	0	2	17
It is OK to make clinical mistakes using the model	1	5	2	5	7
I receive useful educational feedback from the training sessions	0	0	0	1	19
Teamwork is improved by using the central line model	0	1	2	1	16
The central line model simulates central line procedures realistically	0	0	1	6	13
Practice with the central line model boosts my clinical self-confidence	0	0	0	6	14
Practice with the central line model has more educational value than patient care experience	4	2	5	3	6
The training staff is competent	0	0	0	1	19
Practice sessions during the training are a good use of my time	0	0	0	3	17
Practice sessions using procedural models should be a required component of nephrology fellowship	0	0	0	3	17
Repetitive practice using models is a valuable educational experience	0	0	0	6	14
Practice sessions using models are hard work	6	6	1	6	1
Increasing the difficulty of simulated clinical problems helps me become a better doctor	0	0	1	8	11
The controlled environment during the training helps me focus on clinical education problems	0	1	0	5	14
Clinical problems presented during the training are engaging	0	0	0	12	8
Practice with the central line model has helped prepare me to perform the procedure better than clinical experience alone	0	0	0	6	14

Discussion

This study of attending nephrologists and their trainees shows wide variability in THDC insertion skills, with only a small minority of attending nephrologists able to competently insert a THDC at baseline. These findings are concerning but also, reminiscent of the wide variability in skills among attending surgeons performing laparoscopic gastric bypass surgery (17) and multiple studies showing that traditional clinical training does not reliably ensure that physicians can competently perform required procedures (6,7,18–20). Our results are particularly troubling, because they show that attending nephrologists are not adequately trained to supervise and teach fellows in THDC insertion, which is a required skill established by the Royal College of Physicians and Surgeons of Canada (2). Our study findings suggest that skill assessment and training may be needed for attending nephrologists who perform and supervise this procedure.

Despite reporting relatively more experience inserting THDCs, fellows’ overall performances were similar to attending nephrologists’ performances at baseline. Fellows reported very high satisfaction with the SBML intervention, and their THDC skills improved dramatically at post-test. This is similar to the results of earlier studies (6,12,13,18,19,21) and consistent with others showing that SBML improves patient care outcomes and reduces associated health care costs (10–13,22–25).

We believe that the large variation in THDC insertion skills shown in this study is multifactorial. However, one primary factor may be that nephrologists perform fewer actual THDC insertions in contemporary clinical practice because of the increasing referral of procedures to interventional radiology (IR) (26). Referrals to IR do little in equipping nephrologists to practice competently in a situation where emergent dialysis access is required or at sites where radiology support may not be readily available. Additionally, studies show that bedside procedures are less expensive and equally as safe as IR procedures (22,23). Because of the results of this study and others, we recommend rigorous simulation-based training for clinicians who insert THDCs. SBML results in improved THDC insertion skills that are retained over time, translates into the clinical environment (14), and is cost effective (24,27). We previously showed that this training can be provided for a large number of clinicians as part of an annual nephrology meeting (7).

Certain demographic factors correlated with better performance on baseline skills testing. In contrast to earlier work, higher procedural self-confidence was associated with higher simulated THDC insertion skills in this study (6,7,12,18–21). Nevertheless, the vast majority of attending nephrologists and all fellows failed to meet the MPS at baseline, indicating poor overall ability to self-assess procedural skill. This study also showed that younger age and fewer years as an attending nephrologist correlated with better performance on baseline assessment. This confirms earlier work showing that physicians with longer practice experience provided lower quality of care to patients (28). Younger attending nephrologists may be more familiar with ultrasound use during IJ THDC insertion than older attending nephrologists, which may have led to improved checklist performance. It is also possible that THDC skills deteriorate over time when physicians are mostly supervising and not personally performing the procedure (14). Additional study is needed to determine the best mechanism to prepare attending nephrologists to perform and supervise THDC insertions in clinical practice.

This study has several limitations. First, although representing three hospitals, faculty and fellows were all affiliated with a single quaternary care academic center. Although results from fellows in this study are similar to results from previous studies at other academic institutions (6,7), we cannot draw conclusions about the performance of attending nephrologists at other institutions. Second, all attending nephrologists in this study were volunteers, thus potentially introducing volunteer bias. Nonetheless, any volunteer bias likely reinforces our findings, because nonparticipating attending nephrologists might potentially perform worse than volunteers. Third, the checklist used for the simulation-based assessment may not correlate with skills performance on actual patients, because some steps in the simulated environment may lack fidelity (e.g., real patients may have anatomic variations in their vein location and size, have shorter necks, and be moving during the procedure). However, earlier work has shown excellent correlation between checklist scores and performance of actual THDC insertion during clinical practice (14).

In conclusion, this study shows that both attending nephrologists and their trainees had highly variable THDC insertion skills and performed very poorly on objective testing. Given the potential for life-threatening complications related to THDC insertion (3), and the need to offer rigorous training that meets new competency-based education requirements (29), assessment and training for both fellows and attending nephrologists are needed. SBML is an effective method to boost THDC insertion skills and should be considered for attending nephrologists currently performing and supervising this procedure.

Disclosures

None.