Abstract
Background/Aims
To improve procedural skill competence in real-time ultrasound-guided renal biopsy, we have developed an inexpensive simulation tool (a porcine kidney inserted under turkey breast) that mimics biopsy conditions in human patients in terms of kidney size, depth, echogenicity and overall structural characteristics. This study investigated the utility of this simulation tool for improving trainees’ confidence in performing renal biopsy and biopsy-associated bleeding complications.
Methods
We have quantitatively assessed the confidence level of renal fellows before and after their initial renal biopsy simulation training. Subsequently, we determined the effect of this simulation training on trainees’ procedural competence by comparing outcomes of clinical renal biopsies performed by fellows that did vs those who did not participate in the simulation training.
Resuts
We show that the renal biopsy simulation has improved the confidence level of trainees (23.4 pre-simulation to 70.3 post-simulation on a 0–100 scale; p=0.001; eta squared=0.69). The improvement in trainees’ confidence did not vary across their prior experience performing renal biopsies (eta squared=0.23, p=0.060). Additionally, fellows who participated in the simulation training demonstrated improved competence in performing the renal biopsy procedure in patients. Successful retrieval of renal tissue per pass was 94% (vs 73 % in fellows who did not participate in this simulation training; p=0.002) and procedure-related blood loss was reduced as indicated by smaller post-biopsy vs pre-biopsy hematocrit decline (1.18 vs 2.68; p=0.049).
Conclusion
Renal biopsy simulation training may improve trainees’ confidence and reduce the severity of biopsy-associated bleeding complications in patients.
Keywords: kidney biopsy, training, model, bleeding complications
INTRODUCTION
The first surgical simulations were introduced over two decades ago.1 Due to increasing work hour restrictions and the rising prevalence of laparoscopic procedures, simulation experiences have become an integral part of training for surgical residents. Several randomized controlled trials have demonstrated improved surgical resident performance in the operating room when simulation is used to supplement traditional surgical residency training.2–4 As a result of surgical simulation successes, this form of training has spread into medical subspecialties and, effective July 2012, simulation training became one of the Accreditation Council for Graduate Medical Education (ACGME) requirements for Nephrology Training Programs.5
Percutaneous kidney biopsy is an important tool in the evaluation and management of renal disorders. Consequently, the ACGME program requires that both Pediatric and Internal Medicine nephrology fellows develop competent biopsy skills for autologous and transplanted kidneys through formal instruction and clinical experience.5 However, as an invasive procedure, renal biopsy can be associated with significant adverse, even lethal outcomes (reviewed in 6–8). To improve renal fellows’ procedural skill competence in renal biopsy, we have developed an inexpensive, logistically straightforward simulation tool (a porcine kidney inserted under turkey breast) that mimics the characteristics of renal biopsy conditions in human patients in terms of kidney size, depth, tissue echogenicity and overall structural characteristics, and is thus suitable for training of real-time ultrasound-guided renal biopsy.9 The use of this tool for initial renal biopsy training, as well as for maintenance of already acquired skills, has received overwhelmingly positive feedback from fellows in major adult and pediatric nephrology training programs.
To formally evaluate the utility of this simulation tool for improving trainees’ confidence in performing renal biopsy, we have quantitatively assessed the confidence level of renal fellows before and after their initial renal biopsy simulation training. Subsequently, we determined the effect of this simulation training on trainees’ procedural competence by comparing outcomes of clinical renal biopsies performed by a cohort of fellows that did vs those that did not participate in the simulation training.
METHODS
Real-time ultrasound guided renal biopsy simulation
After an initial review of the basic principles of ultrasonographic guidance and renal biopsy procedures, the fellows observed an experienced operator perform several real-time ultrasound-guided biopsies using a simulation tool (a porcine kidney/turkey breast phantom).9 After that, individual fellows practiced real-time ultrasound-guided renal biopsy procedure repeatedly until they attained reasonable accuracy and gained confidence. Guidance and feedback were provided in real time by the experienced operator.
This simulation was performed using a standard ultrasound imaging instrument (Diagnostic ultrasound system Aplio Model SSA-770A; Toshiba Medical Systems Corporation, Tochigi-Ken, Japan). Biopsies were performed with semi-automatic needles (Bard MaxCore Instrument with 18G, 20 cm long needle, 22 mm through; C. R. Bard, Inc., Murray Hill, NJ).
Evaluation of trainees’ confidence in performing renal biopsy
The confidence of renal fellows was evaluated before and after the simulation training. This evaluation was based on pre-simulation and post-simulation surveys completed by all first year nephrology fellows participating in the annual renal biopsy simulation training during the initial 2 months of their training (academic year 2010–2012). In both surveys, the fellows were asked to rate their level of comfort performing a real-time ultrasound-guided renal biopsy under the supervision of an attending on a five-point scale (1 to 5). For easier presentation of the results, the scores were transformed to a 0–100 scale for analysis and reporting.
Evaluation of trainees’ clinical competence in performing renal biopsy
The effect of this simulation training on trainees’ procedural competence was evaluated by comparing outcomes of renal biopsies performed by fellows before and after the implementation of the simulation training into the curriculum in 2010. To eliminate extraneous sources of variability and maximize the equivalence of the two groups, only biopsies performed under the supervision of the same attending during the fellows’ first month of performing renal biopsy procedures were included. Thus, pre-simulation data consist of 21 biopsies conducted by 8 fellows during 2006–2009, and post-training data include 32 biopsies performed by 10 fellows in 2010–2011. All biopsies were performed with a similar ultrasound-guided technique using the Toshiba Diagnostic ultrasound system Aplio Model SSA-770A, Bard Max Core 16G semiautomatic needle, and were supervised by the same attending physician. The biopsy was performed only on those patients who had low risk for bleeding complications, as indicated by coagulation profile that was obtained together with complete blood count prior to the renal biopsy procedure. The number of needle passes and renal tissue cores was documented for every biopsy. Post-biopsy vital signs of these patients were closely monitored and follow up hematocrit values obtained.
To evaluate the effects of simulation training on procedure-related blood loss, changes in pre- and post- procedure laboratory data that are routinely obtained as a part of the pre-biopsy and post-biopsy evaluation were obtained. These data include hematocrit (Hct), as well as coagulation profile (prothrombin time and partial thromboplastin time). The study was approved by the University of Alabama at Birmingham Institutional Review Board,
Statistical analyses
Statistical evaluations were performed with SPSS 11.5 and SAS 9.1 statistical software packages. Changes in fellows’ confidence from before to after the simulation training were analyzed with the nonparametric Wilcoxon Signed Rank test. The effects of previous renal biopsy experience and the effectiveness of training across levels of prior experience on fellows’ confidence were tested with a 2(experience) × 2(time) mixed ANOVA. Before analyzing biopsy success and patient outcomes for fellows who did vs. who did not receive the simulation training, we first tested patient and fellow differences across the two simulation groups with Chi-squared tests of association for categorical variables (patient and fellow gender, intravenous fluid administration and inpatient vs outpatient status) and with general linear models for continuous variables (patient age, PT, INR, PTT, pre-biopsy Hct). Biopsy outcomes of trainees who did vs did not receive the simulation training were analyzed with mixed general linear models predicting biopsy success and differences in pre- vs post-biopsy hematocrit, adjusting for clustering of biopsies within individual fellows. The biopsy success analysis included fellow gender as a covariate. The Hct analyses included pre-biopsy levels of Hct as a predictor.
RESULTS
Real-time renal biopsy simulation improves trainee’s comfort level
To determine the simulation training effects on trainees’ comfort level, we compared trainees’ comfort level with performing renal biopsy before and after the simulation training (n=16). Comparing the pre-simulation (pre-test) to post-simulation (post-test) responses, trainees gained 46.9 points in their confidence level (on a 0–100 scale), increasing from 23.4±7.7(SEM) to 70.3±4.0 [z=(−3.34); p=0.001; Figure 1a].
Figure 1. Real-time renal biopsy simulation improves trainees’ confidence.
(a) Comparing the pre-simulation (pre-test) to post-simulation (post-test) responses, trainees (n=16) gained 46.9 points in their confidence level (0-100 scale), increasing from 23.4±7.7(SEM) to 70.3±4.0 (p=0.001). (b) Fellows’ confidence was improved with the simulation training regardless of prior experience performing renal biopsy procedure, indicated by a non-significant interaction between experience and time (pre vs post-simulation; F[1,14]=4.18; p=0.060; eta squared 0.23; grey shading corresponds to pre-simulation and black shading to post-simulation responses).
Because four fellows (25%) had previously performed renal biopsies (all ultrasound-guided), we further tested whether improvement in confidence differed between those with and without previous renal biopsy experience using a 2(experience) × 2(time) mixed ANOVA. Consistent with the Wilcoxon test, fellows’ confidence improved with training (F[1,14]=30.82; p<0.001; eta squared=0.69). Previous experience had no effect on overall confidence (F[1,14]=1.11; p=0.311; eta squared=0.07). Additionally, fellows’ confidence was improved with the simulation training regardless of prior experience performing renal biopsy procedure, as indicated by a non-significant interaction of experience and time (F[1,14]=4.18; p=0.060; eta squared 0.23; Figure 1b).
Real-time renal biopsy simulation improves trainees’ competence and reduces biopsy-associated blood loss
To examine the effects of the simulation training on fellows’ competence and patients’ outcomes, we compared clinical data on renal biopsies conducted by fellows before and after the ultrasound-guided renal biopsy simulation training was implemented by the program. To reduce variability, these analyses only included a subset of biopsies performed by renal fellows during the first month of performing this procedure, using identical ultrasound-guided technique and supervision by the same attending physician (total 53 biopsies conducted by 18 fellows, 21 pre-simulation and 32 post-simulation; average 3 biopsies per fellow, range 1–8). Two competence outcomes were utilized: 1) the success rate in retrieving the renal biopsy core (number of retrieved renal biopsy core pieces divided by the number of passes); and 2) procedure-related blood loss, reflected by declines in hematocrit levels after the biopsy (there was no major bleeding complication in this group of patients).
First, we compared whether the two groups of fellows who did (n=10) and did not receive (n=8) the simulation training differed in fellow and patient characteristics. The two groups differed in fellow gender [94% vs 62% male; Chi-squared(1)=8.40;p=0.004], but not in any patient characteristics: patient gender [34 vs 29% male; Chi-squared(1)=0.20; p=0.658], age [46 vs 41 yrs; t=−(1.15); p=0.268], prothrombin time [PT; 13.8 vs 13.9; t=0.38; p=0.707], INR [1.04 vs 1.10; t=1.15; p=0.270], partial tromboplastin time [PTT; 28 vs 30; t=1.12; p=0.280], and hematocrit [Hct; 33.5 vs 32.4; t=(−0.59); p=0.565], patient BMI [28.7 vs 26.3; t=1.33, p=0.189], intravenous fluid administrations (6.25% vs 9.52%, Chi-squared(1)=0.20; p=0.659], and outpatient (vs inpatient) status (72% vs 62%, Chi-squared(1)=0.579; p=0.447)]. Because the two groups differed in fellow gender, we examined whether fellow gender was related to any outcomes. Overall, male fellows had higher success rates than female fellows [0.90 vs 0.69; t=(−2.29); p=0.036]. However, there were no gender differences in patient Hct declines (males vs females: Hct 1.81 vs 2.32, t=0.53, p=0.604). Thus, only the analyses of success rate included fellow gender as a covariate.
After adjusting for fellow gender, fellows who received the simulation training had higher success rates in retrieving biopsy core pieces per pass than those who did not receive the training [0.94 vs 0.73 pieces per pass; t=(−3.75); p=0.002; Figure 2a]. The post- vs pre-biopsy Hct decline was significantly lower for fellows who received the simulation training (1.18 vs 2.68; t=2.13; p=0.049; Figure 2b).
Figure 2. Real-time renal biopsy simulation improves trainees’ competence and reduces biopsy-associated blood loss.
(a) Fellows who received the simulation training had higher success rate in retrieving biopsy core pieces per pass than those who did not receive the training [0.94 ± 0.05 vs 0.73 ± 0.04 pieces per pass; t=(−3.75); p=0.002]; based on 53 biopsies performed by fellows who did not (21 biopsies) vs did (32 biopsies) participate in the simulation training.
(b) Fellows who received the simulation training had lower post- vs pre-biopsy Hct decline (1.18 ± 0.49 vs 2.13 ± 0.50; t=2.13; p=0.049). There was no major bleeding complication in this group of patients.
Together, these data suggest that the renal biopsy simulation training using a pork kidney/turkey breast phantom9 improves trainees’ confidence and reduces the severity of biopsy-associated bleeding complications in patients.
DISCUSSION
We have demonstrated that a renal biopsy simulation training using a pork kidney/turkey breast phantom improved trainees’ confidence and reduced the severity of biopsy-associated bleeding complications. These observations are consistent with the effects of other simulation-based medical education approaches that allow the learner to acquire confidence in specific clinical skills by practicing such skills in simulated preselected clinical scenarios10. Surprisingly, we noticed improvement in confidence also among trainees who had prior experience with ultrasound guided renal biopsy procedures. This finding may be explained by the relatively quick confidence loss that occurs among experienced operators required to perform a difficult task11. Similarly, our observation of male fellows having higher success rates in retrieving a biopsy core per pass during ultrasound-guided renal biopsy is consistent with more developed perceptual abilities in males12. However, in the current study, this gender difference in biopsy success rates did not translate to post-biopsy complications, as reflected by post- vs pre- biopsy Hct decline.
A major limiting factor of the present study is the relatively small number of fellows being studied, which limited statistical power to detect smaller effects and potentially introduced selection bias. Additionally, it did not allow the inclusion of potentially important covariates, such as the academic year of each fellow class. However, enrolling more fellows into the study would require extending the study by several years. Alternatively, fellows could be recruited from multiple training programs, but this approach also has important limitations. For example, a substantially higher number of fellows would be required to compensate for training center-specific variability in didactic approach and kidney biopsy technique (e.g., biopsy needle type, gauge, desired number of tissue pieces and target pre-biopsy clinical indices such as BP or INR), as well as site differences in fellow and patient characteristics. Since US nephrology training programs have on average 3–4 new fellows per year, a large number of programs would have to participate in the study, introducing large amount of extraneous variability as well as challenges with project coordination, quality assurance, and data security. In addition, an ideal design should include random assignment of fellows to simulation training (experimental) group and no simulation training (control) group. These issues, together with the ACGME requirement to integrate simulation into the training curriculum, prompted us to evaluate and report the effects of renal biopsy simulation training at a single, homogenous setting using a sample that is relatively small, yet provides sufficient statistical power to detect clinically meaningful effects.
In summary, we have studied the effects of simulation training using a pork kidney/turkey breast phantom. We observed that such training improves trainees’ confidence in performing real-time ultrasound guided biopsy and that the fellows’ confidence has improved regardless of prior experience performing renal biopsy procedure. Additionally, the simulation training improved fellows’ competence in performing the renal biopsy procedure in patients, as reflected by higher likelihood of successful retrieval of renal tissue per pass and reduction of post-biopsy Hct decline. Together, these data suggest that a turkey breast/pork kidney phantom-based simulation training of real-time ultrasound guided renal biopsy improves trainees’ confidence and reduces the severity of biopsy-associated bleeding complications in patients.
Table 1.
Trainees' experience with ultrasound-guided procedures
| Number of procedures | Renal biopsy | Central venous cannulation |
|---|---|---|
| 0 | 75% | 6% |
| 1 – 5 | 19% | 19% |
| 6 – 10 | 6% | 13% |
| > 10 | - | 62% |
ACKNOWLEDGEMENTS
The authors gratefully acknowledge Dr. Michael Allon for critical review of this manuscript. This work was supported in part by the UCSD O'Brien Center 1P30 DK079337 (MM), K01DA024700 (SM) and the Department of Defense and the National Institutes of Health T32DK007695 (JJB). A portion of this work was submitted as an abstract for the National Kidney Foundation Annual Meeting 2012.
Footnotes
There is no conflict of interest.
Conflict of interest: None.
DISCLOSURES
None.
REFERENCES
- 1.Albani JM, Lee DI. Virtual reality-assisted robotic surgery simulation. Journal of endourology / Endourological Society. 2007;21:285–287. doi: 10.1089/end.2007.9978. [DOI] [PubMed] [Google Scholar]
- 2.Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Annals of surgery. 2002;236:458–463. doi: 10.1097/00000658-200210000-00008. discussion 463-454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Uccelli J, Kahol K, Ashby A, Smith M, Ferrara J. The validity of take-home surgical simulators to enhance resident technical skill proficiency. American journal of surgery. 2011;201:315–319. doi: 10.1016/j.amjsurg.2010.08.028. discussion 319. [DOI] [PubMed] [Google Scholar]
- 4.Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. The British journal of surgery. 2008;95:1088–1097. doi: 10.1002/bjs.6344. [DOI] [PubMed] [Google Scholar]
- 5.Winters W, McDonald R, Krauter L. The detection of murine autosomal recesssive polycystic kidney disease using real-time ultrasound. Pediatr Nephrol. 1997;11:337–338. doi: 10.1007/s004670050290. [DOI] [PubMed] [Google Scholar]
- 6.Mackinnon B, Fraser E, Simpson K, Fox JG, Geddes C. Is it necessary to stop antiplatelet agents before a native renal biopsy? Nephrol Dial Transplant. 2008;23:3566–3570. doi: 10.1093/ndt/gfn282. [DOI] [PubMed] [Google Scholar]
- 7.Mackinnon B, McKinlay J, McQuarrie E, Geddes C. Early ultrasound to detect complications after renal biopsy. Nephrol Dial Transplant. 2009 doi: 10.1093/ndt/gfp540. [DOI] [PubMed] [Google Scholar]
- 8.Waldo B, Korbet SM, Freimanis MG, Lewis EJ. The value of post-biopsy ultrasound in predicting complications after percutaneous renal biopsy of native kidneys. Nephrol Dial Transplant. 2009;24:2433–2439. doi: 10.1093/ndt/gfp073. [DOI] [PubMed] [Google Scholar]
- 9.Mrug M, Bissler JJ. Simulation of real-time ultrasound-guided renal biopsy. Kidney Int. 2010;78:705–707. doi: 10.1038/ki.2010.213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does Simulation- Based Medical Education With Deliberate Practice Yield Better Results Than Traditional Clinical Education? A Meta-Analytic Comparative Review of the Evidence. Acad Med. 2011;86:706–711. doi: 10.1097/ACM.0b013e318217e119. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Byrne AJ, Blagrove MT, McDougall SJP. Dynamic confidence during simulated clinical tasks. Postgrad Med J. 2005;81:785–788. doi: 10.1136/pgmj.2004.029942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Coy K, McDougall H, Sneed M. Issues regarding practical validity and gender bias of the Perceptual Abilities Test (PAT) Journal of dental education. 2003;67:31–37. [PubMed] [Google Scholar]