Abstract
Purpose: Ultrasonic propulsion to reposition urinary tract calculi requires knowledge about ultrasound image capture, device manipulation, and interpretation. The purpose of this study was to validate a cognitive and technical skills curriculum to teach urologists ultrasonic propulsion to reposition kidney stones in tissue phantoms.
Materials and Methods: Ten board-certified urologists recruited from a single institution underwent a didactic session on renal ultrasound imaging. Subjects completed technical skills modules in tissue phantoms, including kidney imaging, pushing a stone through a translucent maze, and repositioning a lower pole calyceal stone. Objective cognitive and technical performance metrics were recorded. Subjects completed a questionnaire to ascertain face and content validity on a five-point Likert scale.
Results: Eight urologists (80%) had never attended a previous ultrasound course, and nine (90%) performed renal ultrasounds less frequently than every 6 months. Mean cognitive skills scores improved from 55% to 91% (p<0.0001) on pre- and post-didactic tests. In the kidney phantom, 10 subjects (100%) repositioned the lower pole calyceal stone to at least the lower pole infundibulum, while 9 (90%) successfully repositioned the stone to the renal pelvis. A mean±SD (15.7±13.3) pushes were required to complete the task over an average of 4.6±2.2 minutes. Urologists rated the curriculum's effectiveness and realism as a training tool at a mean score of 4.6/5.0 and 4.1/5.0, respectively.
Conclusions: The curriculum for ultrasonic propulsion is effective and useful for training urologists with limited ultrasound proficiency in stone repositioning technique. Further studies in animate and human models will be required to assess predictive validity.
Introduction
As new technologies are adopted in urology, adequate training must keep pace to ensure safe and effective outcomes for our patients. One potentially transformative technology in the field of urolithiasis management is ultrasonic propulsion. This technology delivers targeted noninvasive acoustic force through focused pulses to move kidney stones. Safety and effectiveness of the technology to reposition stones has been demonstrated in an animal model.1–3 Potential applications of this technology in the management of upper tract stones include facilitating passage of small stones or residual stones after lithotripsy, pre- or intraoperative repositioning of renal stones, and moving an obstructing stone to a nonobstructing position.4
A potential barrier to the adoption of this technology by urologists may be the skill required to operate a diagnostic ultrasound imager and interpret ultrasound images. Experience from animal studies with this technology has shown that effective targeting to push stones requires user skills, including an understanding of the spatial anatomy of the collecting system, visualization of the stone, and alignment of the acoustic force with the desired stone trajectory. Not all urologists perform renal ultrasounds in the office, although it is expected that most urologists would be familiar with interpreting renal ultrasound images.
To optimize the success of this technology, we developed a cognitive and technical curriculum for practicing urologists. The purpose of this study was to test content and face validity of a curriculum to instruct urologists how to perform ultrasonic propulsion of kidney stones in a phantom model.
Materials and Methods
We performed a prospective, institutional review board-approved study to validate our ultrasonic propulsion skills curriculum. Ultrasonic propulsion was performed with a custom-built clinical prototype, including an ultrasound imaging transducer (ATL/Philips C5-2 probe), Verasonics Ultrasound Engine (Redmond, WA), computer processor, and touchscreen, as previously described.2,4 The user identifies a stone under B-mode ultrasound. Touching the stone on the monitor triggers delivery of the acoustic push to the targeted location. Movement of the stone is observed in B-mode in real time.
The curriculum was developed for urologists with the following objectives: (1) acquire knowledge about ultrasound imaging, (2) perform kidney ultrasound and identify kidney stones, and (3) move kidney stones with focused ultrasound. The curriculum was designed to be 1:1 with an instructor and requires 90 minutes in total. The curriculum was divided into three parts, consisting of cognitive skills training, technical skills training, and assessment.
The cognitive skills portion included a didactic presentation and an orientation to the technology. The learners first completed a pretest on the basics of B-mode and Doppler ultrasound. A 10–15-minute review was then given consisting of a discussion on B-mode and Doppler ultrasound, device controls, transducer types, techniques for kidney and kidney stone imaging, and the general concepts of ultrasonic propulsion.
Technical skills training was performed using four modules over ∼45 minutes.
Task 1: Kidney and kidney stone imaging in a human phantom
The goal of this task was to learn ultrasound system controls to optimize kidney and kidney stone imaging in a modified training phantom (Model #BPRB-2011; Blue Phantom, Redmond, WA). The phantom was constructed using simulated tissue to match acoustic characteristics of human tissue under ultrasound imaging and has the appearance of a human torso (Fig. 1A, B). Constructed to match the acoustic attenuation of human tissue, the phantom incorporates adjacent organs, including the liver, ribs, and a modified kidney (Fig. 1C) to include a collecting system space and a 4.0-mm artificial stone. Ultrasound imaging is performed on a tissue cutout from the right flank. Learners were instructed how to hold and orient the transducer and obtain longitudinal and transverse views of the kidney.
FIG. 1.
Ultrasound training modules. Front (A) and back (B) views of modified training phantom with the appearance of a human torso. The tissue cutout from right flank contains an artificial kidney (C) with a collecting system space and artificial stone that matches the acoustic characteristics of a human kidney. The translucent water bath model (D) contains a suspended stone (black arrow) within degassed water to allow the learner to observe stone movement visually and under B-mode ultrasound. Similarly, the transparent maze (E) contains a stone (white circle) on a water-filled curvilinear path (white line). This phantom also simulates tissue to match the acoustic attenuation of human tissue.
Task 2: Suspended stone in custom-built water bath
To demonstrate the concept of ultrasonic propulsion, learners visualized a suspended stone in a water bath (Fig. 1D) under B mode, targeted the stone with the touchscreen, and observed stone movement under B mode and visually without ultrasound. Learners appreciated how the direction of force is generated away from the transducer, and varying the positional alignment between the transducer and the stone can control the direction of the stone motion.
Task 3: Stone in optically transparent maze
With the objective of training how to orient the probe to move the stone in a desired direction, this exercise required learners to direct a 4-mm stone along a two-dimensional, water-filled curvilinear path using ultrasonic propulsion (Fig. 1E). The phantom was constructed using a simulated tissue to match the acoustic attenuation of human tissue (Plastisol; Caney Creek Molds, Denver, IA). The path was created by pouring the phantom material around 6-mm-diameter tubing and removing the tubing after the material set. Stone position and movement were observed under B-mode ultrasound and visual guidance. Learners repositioned and stabilized the transducer to optimize the alignment of the force vector with the varying path.
Task 4: Reposition the lower pole calyceal stone to renal pelvis
To simulate a clinical scenario, where the stone cannot be seen visually but only by ultrasound, learners repositioned a lower pole calyceal stone to the renal pelvis in the same human phantom as in #1 (Fig. 1A–C). Learners used B mode to identify, target, and observe stone movement, without the assistance of visual cues. In addition to proficiency in ultrasound imaging, this exercise required an understanding of the three-dimensional spatial anatomy of the collecting system. The rationale for this model was to simulate a residual fragment after lithotripsy or preoperative repositioning of a lower pole calyceal stone to the midpole before lithotripsy. The renal pelvis was chosen as the endpoint because the kidney phantom does not have a ureteropelvic junction or ureter.
Finally, a 20–30-minute assessment was performed, which began with a written test on the cognitive skills material and technical skills evaluation. Learners were evaluated on their proficiency in kidney and kidney stone imaging, moving the stone in the transparent maze, and repositioning the lower pole calyceal stone to the renal pelvis in the human phantom (as described in tasks 1, 3, 4 above). A standardized set of instructions was given to the subjects before the start of each task. Learners were graded on task time, total pushes delivered, and effective pushes by a nonblinded evaluator. We defined an effective push as one that resulted in stone movement, and % effective pushes were calculated by dividing by the total pushes delivered. Learners were given one attempt to complete each task. Evaluation of performance on the technical skills portion of the curriculum was completed by a technician experienced with the technology (Ryan S. Hsi). At the completion of the curriculum, each subject completed a questionnaire to ascertain face and content validity based on a five-point Likert scale. Learners were asked to rate statements based on their level of agreement or disagreement.
We enrolled 10 board-certified, academic urologists at a single institution. Demographic data were collected and subjects were excluded if they had previous experience using the technology. Subjects were given $10 in compensation in the form of a gift card.
Descriptive statistics were used to summarize data on demographics, technical skills, and face and content validity. The paired t-test was used to compare cognitive scores before and after the cognitive skills session. Statistical analyses were performed using Stata 12.1 (StataCorp LP, College Station, TX) with values<0.05 considered statistically significant.
Results
The demographic information for the 10 subjects is shown in Table 1. Eight urologists (80%) had never attended a previous ultrasound course, and nine urologists (90%) performed renal ultrasound examinations less frequently than once per 6 months. Because both the fellowship-trained endourologists at our institution had previous experience with the technology, they were excluded from the study (Jonathan D. Harper and Mathew D. Sorensen). Most subjects (70%) spent the majority of their clinical practice managing patients with nephrolithiasis.
Table 1.
Subject Demographics
| Characteristic | n | % |
|---|---|---|
| Age, years | ||
| 30–44 | 3 | 30 |
| 45–59 | 6 | 60 |
| ≥60 | 1 | 10 |
| Median years since residency (IQR) | 19 (9–24) | |
| Gender | ||
| Male | 8 | 80 |
| Female | 2 | 20 |
| Handedness | ||
| Right | 8 | 80 |
| Left | 1 | 10 |
| Both | 1 | 10 |
| Pediatric subspecialty | 3 | 30 |
| Attended prior ultrasound course | 2 | 20 |
| % of clinical practice managing nephrolithiasis | ||
| None | 1 | 10 |
| <5% | 0 | 0 |
| 5%–25% | 1 | 10 |
| 26%–50% | 1 | 10 |
| >50% | 7 | 70 |
| Ultrasound use in clinical practice,a mean | ||
| Interpret/read any ultrasound | 4.3 | |
| Interpret/read renal ultrasounds | 4.2 | |
| Perform/obtain any ultrasound | 2.7 | |
| Perform/obtain renal ultrasounds | 1.5 | |
Graded as 1=I do not perform/obtain ultrasounds in my practice, 2=less than once per 6 months, 3=less than once per month, 4=at least once per month, 5=at least once a week.
All training sessions were completed in less than 90 minutes. Cognitive and technical skills assessment results are shown in Table 2. Mean cognitive skills scores on the written test improved from 55% to 91% (p<0.0001). All learners completed the transparent maze with a mean±SD of 12.5±3.5 pushes and 71%±21% effective pushes. In the kidney phantom, all 10 (100%) subjects repositioned the lower pole calyceal stone to at least the lower pole infundibulum, and 9 of 10 (90%) successfully repositioned the stone to the renal pelvis. The remaining subject who did not complete the task had difficulty stabilizing the transducer while using the other hand to target the stone for pushing using the touchscreen. Overall, an average of 15.7±13.3 pushes with 80%±15% effective pushes were required to complete the kidney phantom task.
Table 2.
Cognitive and Technical Skills Assessment Results
| Task | |
|---|---|
| Cognitive assessment,a mean±SD (%) correct | |
| Pretest | 56%±17% |
| Post-test | 91%±11% |
| B-mode imaging | |
| Optimized adjustment of ultrasound controls | 100% |
| Correct transducer orientation | 100% |
| Obtained transverse and longitudinal views | 100% |
| Identified kidney stone and location | 100% |
| Translucent maze | |
| Time (minutes), mean±SD | 4.6±2.2 |
| Total pushes delivered, mean±SD | 12.5±3.5 |
| % Effective pushes delivered, mean±SD (%) | 71%±21% |
| Successful task completion | 100% |
| Human phantom | |
| Reposition lower pole calyceal stone | |
| To infundibulum | 100% |
| To renal pelvis (goal) | 90% |
| Time (minutes), mean±SD | 4.5±3.1 |
| Total pushes delivered, mean±SD | 15.7±13.3 |
| % Effective pushes delivered, mean±SD (%) | 80%±15% |
p<0.0001.
Urologists rated the curriculum's effectiveness and realism as a training tool at mean scores of 4.6/5.0 and 4.1/5.0, respectively (Table 3). All subjects rated agree or strongly agree for relevance of the technology to the practice of urology, and whether urologists should undergo training before clinical use.
Table 3.
Questionnaire Results
| Mean Likert scores (maximum score of 5)a | |
|---|---|
| Model feedback | |
| Helpfulnessa | |
| Water bath | 4.2 |
| Transparent maze | 4.7 |
| Imaging kidney stones in human phantom | 4.3 |
| Pushing kidney stones in human phantom | 4.7 |
| Ease of useb | |
| Water bath | 4.4 |
| Transparent maze | 3.4 |
| Imaging kidney stones in human phantom | 3.6 |
| Pushing kidney stones in human phantom | 2.9 |
| Validationc | |
| Content validation—the curriculum was effective for training in | |
| Imaging the kidney | 4.6 |
| Imaging kidney stones | 4.3 |
| Pushing kidney stones | 4.8 |
| Face validation—the curriculum was realistic in | |
| Imaging the kidney | 4.3 |
| Imaging kidney stones | 3.8 |
| Pushing kidney stones | 4.1 |
| The technology is relevant to the practice of urology | 4.9 |
| Urologists should undergo training before using this technology | 4.9 |
Graded as 1=not helpful, 2=neutral, 3=slightly helpful, 4=helpful, 5=very helpful.
Graded as 1=very difficult, 2=difficult, 3=slightly difficult, 4=easy, 5=very easy.
Graded as 1=strongly disagree, 2=disagree, 3=neutral, 4=agree, 5=strongly agree.
Discussion
Creating a stepwise educational method for the introduction of new technology may minimize the learning curve for surgeons and enhance initial clinical outcomes.5 There are several important findings from this study on an ultrasonic propulsion curriculum for urologists. Among urologists with a limited degree of baseline ultrasound proficiency, the curriculum was effective for training stone propulsion technique. All learners completed the transparent maze module successfully with a high proportion of ultrasonic pushes that resulted in effective movement. All learners demonstrated a grasp of being able to move the stones. Most (90%) moved the stones to the targeted renal pelvis, with a high proportion of ultrasonic pushes that resulted in effective movement. Pushing stones in the human phantom model was overall rated as very helpful, but was also rated to be slightly difficult. This suggests that applying the technology to a three-dimensional anatomy system requires skill and further refinements to the phantoms can be made to provide feedback on optimal push direction. These performances may translate into clinically relevant skills as the stone pushing technology may be able to get recalcitrant stones or stone fragments into passable locations within the upper tract. The curriculum received high ratings on the effectiveness and realism of the training simulation.
Developing and validating curricula before the rapid adoption of a new technology are important to establish training and credentialing processes that will ensure the safe and effective use as providers begin using the technology. The view that validation and curriculum development should occur together has been embraced by the American Urological Association (AUA) Laparoscopy and Robotic Surgery Committee for basic robotic surgery training.6 In the best practice statement by Lee et al., the authors conclude that robotic surgery credentialing should be based on a demonstration of proficiency and safety in the training setting rather than the number of completed cases performed.6 The introduction of ultrasonic propulsion as a treatment modality for urinary tract calculi would also be expected to require organized educational curricula and proficiency-based credentialing initiatives.
One method of dissemination of this curriculum may be as a supplement to the existing ultrasound courses to train urologists for bedside or in-office ultrasonography. The AUA and American Institute of Ultrasound in Medicine (AIUM) have created training programs to train urologists in imaging based on practice guidelines for performing ultrasound examinations in urology.7 The guidelines outline specifications for urologic ultrasound examinations, including kidney, bladder, prostate, scrotal, and penile examinations. This concept of the “point-of-care” ultrasound has been defined as “ultrasonography brought to the patient and performed by the provider in real time.”8 Several published reviews have described the utility of incorporating ultrasound examinations for the office urologist9–13 and on-call urologist.14–16 Whereas there is some evidence that suggests urologists do not routinely perform their own renal ultrasound examinations in the United States (<6% of all noncardiac examinations in 2009),17 a dedicated accreditation program sponsored by the AUA for urologists and urology residents called the Hands-on Urologic Ultrasound Course18 has had over 1000 attendees in more than 20 sessions offered since 2006 (AUA Office of Education, pers. comm.).
Our findings must be interpreted within the context of limitations of our study design. First, the assessment instruments used in this study have not been validated, as this technology is novel with recent FDA approval to perform feasibility studies. Therefore, the predictive validity of this study is unknown and requires long-term follow-up.19 Second, the training models used in this study may not represent conditions in the human model, although face and content validation ratings were overall high. For instance, the phantom in this study represented a nonobese patient, and we envision that there will be challenges in using this technology in individuals with increasing BMI. It is anticipated that the data from human trials will provide needed information to enhance these training models. Third, the subjects were recruited from a single academic institution, which may limit the generalizability of these findings to nonacademic urologists. However, we did sample urologists from a broad range of subspecialization (trauma/reconstruction, female, neurourology, oncology, pediatrics) and at various stages of their careers. Finally, as this is a novel technology, what is considered expert and novice is still in the process of being defined. Further developments in the technology are expected to also refine the appropriate technical tasks, assessment, and outcomes needed to characterize proficiency. Feedback from the urologists in this study has been used to enhance and simplify the user interface of the device. In addition, a new human phantom is in development with a larger tissue cutout to allow an expanded window to perform ultrasound on the kidney.
Conclusions
Among urologists with limited ultrasound proficiency, a curriculum to train ultrasonic propulsion to reposition urinary tract calculi is effective and useful. Cognitive scores on ultrasound fundamentals were significantly improved, and all urologists demonstrated the technical skill required to noninvasively reposition a stone out of the lower pole calyx in a human phantom. Further studies in animate and human models will be required to assess predictive validity.
Abbreviations Used
- AIUM
American Institute of Ultrasound in Medicine
- AUA
American Urological Association
Acknowledgments
This work was supported by the NIH DK43881, DK092197, and NSBRI through NASA NCC 9-58.
Disclosure Statement
No competing financial interests exist.
References
- 1.Shah A, Harper JD, Cunitz BW, et al. Focused ultrasound to expel calculi from the kidney. J Urol 2012;187:739–743 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Harper JD, Sorensen MD, Cunitz BW, et al. Focused ultrasound to expel calculi from the kidney: Safety and efficacy of a clinical prototype device. J Urol 2013;190:1090–1095 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Connors BA, Evan AP, Blomgren PM, et al. Comparison of tissue injury from focused ultrasonic propulsion of kidney stones versus extracorporeal shock wave lithotripsy. J Urol 2013. DOI: 10.1016/j.juro.2013.07.087 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Sorensen MD, Bailey MR, Hsi RS, et al. Focused ultrasonic propulsion of kidney stones: Review and update of preclinical technology. J Endourol 2013;27:1183–1186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007;193:797–804 [DOI] [PubMed] [Google Scholar]
- 6.Lee JY, Mucksavage P, Sundaram CP, et al. Best practices for robotic surgery training and credentialing. J Urol 2011;185:1191–1197 [DOI] [PubMed] [Google Scholar]
- 7.Practice Guideline for Urologic Ultrasound Available at: www.auanet.org/education/practice-guideline-for-urologic-ultrasound.cfm (Accessed: June12, 2013)
- 8.Moore CL, Copel JA. Point-of-care ultrasonography. N Engl J Med 2011;364:749–757 [DOI] [PubMed] [Google Scholar]
- 9.Acino S, Resnick MI. Office urologic ultrasound. Urol Clin North Am 1988;15:577–588 [PubMed] [Google Scholar]
- 10.Donovan JM, Ney KG, Maizels M. Urosound. In-office ultrasonography for pediatric urology. Urol Clin North Am 1989;16:841–855 [PubMed] [Google Scholar]
- 11.Goldenberg E, Gilbert BR. Office ultrasound for the urologist. Curr Urol Rep 2012;13:460–466 [DOI] [PubMed] [Google Scholar]
- 12.Leonard SA, Thomas R. Diagnostic ultrasound in the urologist's office. Urology 1987;29:666–668 [DOI] [PubMed] [Google Scholar]
- 13.McAchran SE, Dogra V, Resnick MI. Office urologic ultrasound. Urol Clin North Am 2005;32:337–352 [DOI] [PubMed] [Google Scholar]
- 14.Dalziel PJ, Noble VE. Bedside ultrasound and the assessment of renal colic: A review. Emerg Med J 2013;30:3–8 [DOI] [PubMed] [Google Scholar]
- 15.Moslemi MK, Mahfoozi B. Urologist-operated ultrasound and its use in urological outpatient clinics. Patient Prefer Adherence 2011;5:85–88 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Surange RS, Jeygopal NS, Chowdhury SD, et al. Bedside ultrasound: a useful tool for the on-call urologist? Int Urol Nephrol 2001;32:591–596 [DOI] [PubMed] [Google Scholar]
- 17.Levin DC, Rao VM, Parker L, et al. Noncardiac point-of-care ultrasound by nonradiologist physicians: How widespread is it? J Am Coll Radiol 2011;8:772–775 [DOI] [PubMed] [Google Scholar]
- 18.Ultrasound Accreditation Available at: www.auanet.org/education/ultrasound-accreditation.cfm (Accessed: June12, 2013)
- 19.Van Nortwick SS, Lendvay TS, Jensen AR, et al. Methodologies for establishing validity in surgical simulation studies. Surgery 2010;147:622–630 [DOI] [PubMed] [Google Scholar]