Imaging skills for transthoracic echocardiography in cardiology fellows: The value of motion metrics

Abstract

Background:

Proficiency in transthoracic echocardiography (TTE) requires an integration of cognitive knowledge and psychomotor skills. Whereas cognitive knowledge can be quantified, psychomotor skills are implied after repetitive task performance. We applied motion analyses to evaluate psychomotor skill acquisition during simulator-based TTE training.

Methods and Results:

During the first month of their fellowship training, 16 cardiology fellows underwent a multimodal TTE training program for 4 weeks (8 sessions). The program consisted of online and live didactics as well as simulator training. Kinematic metrics (path length, time, probe accelerations) were obtained at the start and end of the course for 8 standard TTE views using a simulator. At the end of the course TTE image acquisition skills were tested on human models. After completion of the training program the trainees reported improved self-perceived comfort with TTE imaging. There was also an increase of 8.7% in post-test knowledge scores. There was a reduction in the number of probe accelerations [median decrease 49.5, 95% CI = 29-73, adjusted P < 0.01], total time [median decrease 10.6 s, 95% CI = 6.6-15.5, adjusted P < 0.01] and path length [median decrease 8.8 cm, 95% CI = 2.2-17.7, adjusted P < 0.01] from the start to the end of the course. During evaluation on human models, the trainees were able to obtain all the required TTE views without instructor assistance.

Conclusion:

Simulator-derived motion analyses can be used to objectively quantify acquisition of psychomotor skills during TTE training. Such an approach could be used to assess readiness for clinical practice of TTE.

INTRODUCTION

Transformation of graduate medical education from a time-based apprenticeship to a competency-based model has led to several innovations. Recently, haptic (i.e., with tactile and directional feedback) mixed echocardiography simulators have been incorporated in a curriculum-based education model to enhance the efficiency of teaching and evaluate proficiency in transesophageal echocardiography (TEE).[1] Using this technology, acquisition and progression of psychomotor skills for TEE can now be analyzed using kinematic (i.e., motion) metrics. Trainees who underwent an integrated simulator-based curriculum for TEE education also demonstrated some elements of clinical transferability.[2] Similar motion metrics have been successfully incorporated in training, evaluation, and certification in gastrointestinal endoscopy and laparoscopic surgery.[3,4,5,6] Currently, acquisition of psychomotor skills for clinical procedures is not quantified, but rather implied after repetitive clinical exposure. Utilization of kinematic analyses offers a possibility to objectively evaluate psychomotor skills.

While both are cardiac imaging techniques, the process of image acquisition differs significantly between TEE and transthoracic echocardiography (TTE). Invasive nature and the confines of esophagus necessitate minute and subtle probe motions (pitch, roll, and yaw) to obtain adequate TEE imaging windows. In comparison, during a TTE examination, probe motions are not only different in characteristics (sliding, rotation, tilting) but also can have longer trajectories over the chest wall. Various motion metrics, such as magnitude and smoothness of hand motion have demonstrated correlation with acquisition of manual dexterity for several procedures.[7,8,9,10] Whether these metrics can be used to track the progression of TTE skills has not been evaluated. Therefore, using a haptic TTE simulator we evaluated psychomotor skill acquisition during simulator-based TTE training for cardiology fellows.

METHODS

Approval for this study was obtained through our Institutional Review Board, with a waiver of informed consent. The study was performed on two consecutive years (2013–2014). Each year, TTE-naïve 1^st year cardiology fellows undergoing the 1^st month of their fellowship training were invited to participate. Exclusion criteria included: Having undergone previous structured TTE training, previous performance of >5 complete TTE examinations, and possession of certification by the National Board of Echocardiography in TTE.

The participants underwent a multimodal TTE training course, which consisted of eight hands-on sessions imparted twice weekly over 4 weeks. Before each hands-on session, they completed an online module on a dedicated educational website (anesthesiaeducation.net/moodle). The online modules covered the following topics: TTE imaging planes, left ventricular function, right ventricle, mitral valve, aortic valve, hemodynamics, pericardial disease, and TTE in clinical decision-making. The hands-on sessions were 90 min each and consisted of a brief (15 min) lecture reviewing the concepts of the online module, followed by supervised hands-on practice on Heartworks (Inventive Medical, London, England) and Vimedix (CAE Healthcare, Montreal, Canada) TTE simulators.

Knowledge was assessed through a 61-item online questionnaire (available at http://www.anesthesiaeducation.net/moodle/mod/quiz/view.php?id=1656; username = sample, password = sample) before the course and after the conclusion of the course. The trainees were asked to rate their comfort with and self-perceived knowledge of TTE before the course and after completing the course using a 5-point Likert scale (1 = very poor, 2 = poor, 3 = fair, 4 = good, 5 = excellent). In addition, at the end of the course, trainees were asked to rate the utility of the hands-on sessions, lectures, and online modules on the same scale.

Motion metrics were evaluated at the end of the second and eighth training session on two Vimedix (CAE Healthcare, Montreal, Canada) TTE simulators. The metrics program on the simulator was selected. To obtain these data, we used a similar methodology as the one we previously employed for TEE.[2]

Briefly, an experienced echocardiographer acquired eight target cut planes (TCP), or simulator reference images corresponding to a standard TTE view. The following TCP were obtained: Apical 2-, long axis, and 4-chamber; subcostal inferior vena cava, parasternal (PS) aortic valve short axis, PS long axis, PS right ventricular inflow, and PS mid-short axis). These TCP's were then reviewed and approved by all the investigators.

For kinematic analysis the following protocol was followed:

• The TTE probe was placed on the mannequin's abdomen (at the level of the umbilicus) prior to acquisition of each TCP by the trainee to standardize the starting point of image acquisition

• The metrics menu on the simulator was selected for each TCP, which initiated motion data collection

• The trainee was then instructed to obtain a specific TCP

• The kinematic data collection was terminated when the instructor established the acquisition of a satisfactory TCP by the trainee

• The above cycle was repeated for each of the eight TCP's.

The Vimedix simulator is capable of acquiring the probe positional (x, y, and z coordinates) and time data for each TCP at a high sampling rate (several times/second). These data were then exported as a comma separated values file and analyzed using Microsoft Excel (Microsoft, Redmond, WA). The following motion metrics were derived from the positional and time data:

• Time (s): The total time required by the trainee to acquire each TCP from the start to the end of an acquisition cycle

• Path length (cm): The sum of all linear and angular movements of the probe during each imaging cycle

• Probe accelerations from rest (#): The number of times the probe acceleration exceeded 0.5 cm/s².

At the end of the course, participants were instructed to acquire echocardiographic views corresponding to the TCP's in healthy human models. Their success in obtaining the different views was assessed by an instructor, and the quality of the view was rated as poor, moderate, or excellent.

Additional analyses were performed using SPSS version 21 for Mac (IBM, New York, NY, USA). Comparisons were performed with the paired Student's t-test or the Wilcoxon matched pairs signed rank test, where applicable. Adjustment of P values for multiple comparisons was performed for the metrics data using the Holm–Bonferroni sequential correction,[11] and significance was set at <0.05. Confidence intervals of the mean difference for the motion metrics data were obtained by using the Hodges–Lehmann estimator. Data are reported as mean ± standard deviation (SD) or median (interquartile range), where applicable.

RESULTS

Sixteen participants completed the study (n = 9 in year 1, n = 7 in year 2). Pretest knowledge scores (68.1% ± SD 8.7) were significantly lower when compared to posttest scores (76.8% ± SD 5.4; P = 0.002). The perceived comfort with TTE increased significantly from the start to the end of the course (2.2 ± SD 0.7 and 3.7 ± SD 0.5, respectively, P < 0.0001). Self-perceived knowledge of TTE also increased significantly (P < 0.0001) at the end of the course (3.6 ± SD 0.5) when compared to the start of the course (2.3 ± SD 0.8) [Figure 1]. The trainees’ ratings for the usefulness of hands-on sessions, lectures, and online modules were 4.5 ± SD 0.5, 3.9 ± SD 0.8, and 4.0 ± SD 0.8, respectively [Figure 1].

Figure 1.

(a) Pre- and post-test knowledge scores. (b) Comfort with transthoracic echocardiography before and after the course. (c) Self-perceived knowledge of transthoracic echocardiography. (d) Perceived utility of the different elements of the course

The number of probe accelerations from rest, total time, and path length decreased significantly (adjusted P < 0.01 for all) from the start to the end of the course [Table 1]. These improvements were consistent throughout all the TCPs [Figure 2].

Table 1.

Kinematic metrics obtained at the start and end of the course*

Metric	Start of course	End of course	Absolute median difference; 95% CI	Adjusted P
Path length (cm)	64.5 (40.2-103.5)	49.9 (35.7-66.9)	8.8; 17.7- 2.2	<0.01
Probe accelerations (#)	161 (88-306)	90 (52-139)	49.5; 73-29	<0.01
Total time (s)	24.6 (15.7-51.8)	17.0 (12.4- 26.3)	10.6; 15.5-6.6	<0.01

*Median (interquartile range); *P<0.05 is significant. CI: Confidential Interval

Figure 2.

Kinematic metrics for the different target cut planes at the start and end of the course. (AVSAX: Aortic valve short axis; IVC: Subcostal inferior vena cava; MID PAP: Parasternal mid-short axis; PS LAX: Parasternal long axis; PS RV INF: Parasternal right ventricular inflow)

In the assessment on human models, all the trainees were able to obtain all views with at least moderate quality without instructor assistance.

DISCUSSION

Our study has demonstrated that motion metrics can be used to quantify psychomotor skill acquisition in TTE naïve cardiology fellows during simulator training. We found that a significant reduction in the total time and path length for TCP acquisition occurred at the end of the course [Table 1 and Figure 3]. In addition, the cardiology fellows also demonstrated a reduction in the number of probe accelerations (i.e., peak motions) during image acquisition. These data suggest that efficiency and intuitiveness of hand motion (i.e., automaticity) can be assessed during TTE simulator training. There was a reduction in total magnitude and improvement in smoothness of probe motion after the course. This study extends the role of motion metrics that were specifically developed for assessing TEE (path length and probe accelerations) to TTE training. In addition, the trainee's demonstrated elements of clinical transferability after simulator training, acquiring satisfactory TTE images on live models. Our training program was unique in that didactics were integrated with hands-on simulator training, resulting in simultaneous improvement in knowledge and psychomotor skills. While the hands-on training resulted in the highest participant rating, live lectures, and online training were also positively rated, suggesting that the course structure was appropriate for the participants’ training level.

Figure 3.

Three-dimensional reconstruction of the path length for one trainee at the start and end of the course during acquisition of the apical long axis view

Simulation has been used for task training across multiple disciplines to improve operator performance and patient safety.[12,13,14] Simulators facilitate training for complex clinical tasks requiring both cognitive knowledge and manual dexterity, overcoming the limitations that traditional training has for providing a graduated (simple to complex), and integrated educational experience.[15,16]

Since TTE is noninvasive, it can be argued that repetitive exposure to live models without the use of simulator-derived kinematic measures can achieve success in imparting proficiency. However, motion analyses may add an element of objectivity for assessing the readiness of trainees for clinical practice of specific procedures.[17] Motion metrics could identify trainees who need more instruction prior to clinical exposure.[18] These motion analyses have shown construct validity in other disciplines.[2,8,16,18,19]

Our study possesses certain limitations. First, we did not have a control group to validate the acquisition of manual skills during conventional training. However, this was not a study of the value of simulator as a task trainer; we evaluated the use of kinematic measures, and each trainee acted as their own control. Second, our sample size is relatively small, mainly because of our strict inclusion criteria. Third, there is no ideal method for evaluating clinical transferability of simulator-acquired skills. However, certain elements of transferability were demonstrated on live models.

CONCLUSION

Motion analyses can be used to evaluate psychomotor skill acquisition during TTE simulator training. These motion metrics may demonstrate an increased automaticity and efficiency of motion that occurs during psychomotor learning. Integration of these kinematic analyses into a simulator-based multimodal TTE training program may result in a more efficient training, with a shortening of the learning phase prior to clinical exposure.

Financial support and sponsorship

This study was partially funded by a grant from the Foundation for Anesthesia Education and Research.

Conflicts of interest

There are no conflicts of interest.