Objective Assessment of Skill Retention 7 Months Post-Training: Motion Analysis of Central Venous Catheter Placement

Abstract

Background:

Central venous catheter (CVC) placement is a technically challenging skill. Routine assessment tools, including checklists and global rating scales, require subjective expert evaluation. We hypothesized that motion analysis could be used to objectively assess skill retention in CVC placement by comparing the performance of anesthesiology residents immediately after training and 7 months later.

Methods:

After learning to perform CVC placement on a mannikin, 12 first-year anesthesiology residents each performed a “baseline” trial with electromagnetic motion sensors on the dorsum of their dominant hand and base of their ultrasound probe. Seven months later, they each performed a “follow-up” mannikin trial with an identical setup. For each trial, sensors recorded participants’ path length, translational motions, and rotational sum. Time was recorded for each trial as well. We defined skill retention as performance within 1 standard deviation or less of the entire cohort's average at baseline (threshold). We compared the number of residents who met the threshold, which indicated less excessive motion and therefore better performance, at baseline with the number at follow-up using McNemar's test across each metric for each sensor.

Results:

For path length, translational motions, and rotational sum of the probe, significantly more residents met the threshold at baseline than at follow-up (P < .04). No significant differences were detected for any metrics of the dorsum or time.

Conclusions:

Motion analysis can objectively assess skill decay in anesthesiology residents performing CVC placement. Residents exhibited skill retention in tasks involving their dominant hand and skill decay in tasks involving the ultrasound probe (nondominant hand).

Introduction

Central venous catheter (CVC) placement is a common but technically challenging skill that requires fine motor dexterity, hand-eye coordination, and, in line with current standards, a general proficiency with ultrasound (US). CVC placement carries the rare but life-threatening risks of arterial cannulation, hematoma, and pneumothorax.^1,2 Therefore, optimizing training and maintaining competency is essential.¹ Several studies have used checklist assessments and global rating scales (GRSs) in simulation training to assess skill acquisition of CVC placement^3,4 and retention.^5-7 Although these assessment tools are routinely used, they require an expert evaluator and rely on subjective interpretation.^8,9 They also grade on an ordinal scale, meaning scores can be ordered/ranked but are still categorical, as opposed to a continuous scale, obfuscating incremental differences in performance across categorical scores. Therefore, in the context of longitudinal training, the transition from a failing score to a passing score may be poorly appreciated.

Motion analysis allows for continuous, objective measurement throughout the performance of a technical procedure. It has the potential to better define competency-based training and more precisely assess the incremental achievement of milestones.⁸ Specific to central line placement, motion metrics have been demonstrated to objectively differentiate novices and experts⁹ and assess learning curves in skill acquisition.¹⁰ Motion metrics may serve to supplement expert observation, checklists, and GRSs to provide more personalized objective feedback on specific fine motor skills, as previously described.^11,12 Furthermore, segmentation of motion metric data into discrete subtasks for a procedure may expedite focused remediation of component elements, as opposed to retraining the entire procedure.¹¹

Skill decay is an omnipresent and universally relevant phenomenon in medical education, yet there remains a dearth of methods to objectively assess it.⁷ A recent meta-analysis demonstrated that skill decay, as measured by checklists and GRSs, could be observed as early as 3 months after training and that performance tended to decline throughout the 12 months following.⁷ Notable variability among reported studies prompted this investigation into objective measures of decay. The use of motion analysis to study skill decay has been demonstrated as feasible and reliable with various skill sets.^13,14 We therefore hypothesized that motion analysis could be used to objectively assess skill retention in CVC placement by comparing performance of anesthesiology residents immediately after training and 7 months later. As a secondary hypothesis, we sought to determine whether segmentation analysis could identify skill retention or decay in isolated tasks of CVC placement.

Methods

This prospective cohort study received institutional review board approval for exempt status with a waiver of documentation of informed consent by the Committee on Clinical Investigations at Beth Israel Deaconess Medical Center.

Original Course Logistics

Twelve first-year anesthesiology residents (postgraduate year 1 interns) underwent a previously described 13-day basic anesthesia/US course led by senior residents, fellows, and attending anesthesiologists.¹⁵ For 2 days of this course, the residents, of whom 11 were right-handed and 1 was left-handed, learned to perform CVC placement as described in our previous study.¹¹ At the conclusion of the second day, residents performed a final CVC trial on a SimuLab (SimuLab Corporation, Seattle, WA) CentraLine System mannikin using a Butterfly iQ+ (Butterfly Network, Inc, Guilford, CT) US probe connected to an Apple iPad Mini (Apple Inc, Cupertino, CA) with a 7.9-inch screen. During this trial, their motions were recorded by applying electromagnetic sensors to the dorsum of residents’ dominant hands and base of their US probe in a standardized procedural setup (Figure 1). The equipment, sensors, mannikin, and setup used in this study were identical to those used in our previous study where we tracked the same motions for novice anesthesiology interns and expert attending anesthesiologists.¹¹

Figure 1.

(A) Standard procedural setup before each central venous catheter placement trial. (B and C) Electromagnetic sensors were placed on the base of the ultrasound probe (B) and center of the dorsum (C). This figure has been reproduced from a previous publication.¹¹

Course Follow-Up

Seven months following the course (when the interns started their first year of residency), without formal intermediate US training on simulated or human patients, residents underwent a 4-hour introduction-to-US training session led by 2 attending anesthesiologists as part of their orientation to residency. This course was a general introduction to US techniques that consisted of brief lectures on knobology, US physics, US-guided vascular access, and transthoracic echocardiography, followed by hands-on practice on the same SimuLab CentraLine System mannikin, Blue Phantom US training blocks (CAE Healthcare, Inc, Sarasota, FL), and a CAE Vimedix transthoracic echocardiography simulator (CAE Healthcare, Inc, Sarasota, FL). At the station with the SimuLab CentraLine System mannikin, the residents had a brief review of CVC placement and 2 practice trials on the mannikin using the same US equipment (Butterfly iQ+ probe connected to a 7.9-inch Apple iPad Mini) as during the original course. Residents then performed a diagnostic CVC placement trial on the mannikin while their motions were recorded. Recordings from the final trials during the original course (termed “baseline”) and from the 7-month follow-up (termed “follow-up”) were performed in an identical fashion, using the same procedural design, equipment, and setting.

Motion Recordings

To record residents’ motions, participants were equipped with electromagnetic sensors from a Polhemus Liberty (Polhemus, Colchester, VT) motion tracker. Motions were recorded using previously defined methodologies.^8,11 Each trial was video recorded to properly segment motion recordings by checkpoints for composite analysis. Segments were defined as follows based on previous studies¹¹:

• Segment 1 (distinguishing the internal jugular vein from the carotid artery): from a participant's first movement to the needle's first contact with the skin (checkpoint 1)

• Segment 2 (obtaining venous access): from checkpoint 1 to the removal of the hub of the needle following aspiration of venous blood (checkpoint 2)

• Segment 3 (insertion of the guidewire to between 20 and 30 cm, long and short axis US confirmation of wire placement in the vessel lumen by the resident with confirmation by a course instructor, and threading of the dilator to the skin): from checkpoint 2 to threading the dilator to the skin (checkpoint 3)

Statistical Analysis

Motion data were postprocessed using Microsoft Excel (Microsoft Corporation, Redmond, WA). For each trial, 4 previously described metrics were calculated for each sensor¹¹:

1. Path length (centimeters): the total distance a sensor traveled

2. Number of translational motions: the number of times a translational acceleration exceeded a threshold of 15 mm/s² after previously being below it in the prior frame

3. Rotational sum (°): the total degrees of rotation a sensor underwent

4. Time (seconds): the total time

The formula for rotational sum was adjusted to the following to more accurately calculate the changes in Euler orientation angles given that the range of angle values the Liberty motion tracker provides is 0° to ±180° as opposed to 0° to 360°:

$$\sum _{i=1}^n\sqrt{\left(a^2+b^2+c^2\right)}$$

where ψ, Ø, Θ and represent the Euler orientation angles, i represents the frame number, a is defined as

• If _i−1 and are both ≤0 or ≥0, a = |_i - _{i- 1}|

  If _{i- |1} < 0 and _i > 0 or vice versa,

  • ° If |_{i- 1}|+|_i|≤ 180, a = |_{i- 1}|+||i|

  • ° If |_{i- 1}|+|i|> 180,

    a = (180-|_{i- 1}|)+(180- |i|,

    b is defined as

• If Θ_{i -1} and Θ_i are both ≤0 or ≥0, b = |Θ_i–Θ_{i -1}

• If Θ_{i- 1} < 0 and Θ_i > 0 or vice versa,

  • °If |Θ_{i -1} |+|Θ_i| ≤ 180, b = Θ_{i -1} |+|Θ_i|

  • °If |Θ_{i -1} |+|Θ_i| > 180, b = (180– |Θ_{i- 1})+(180– |Θ_i|), and c is defined as

• If Ø_{i- 1} and Ø_i are both ≤0 or ≥0, c = |Ø_i – Ø_{i -1}|

• If Ø_{i -1} < 0 and Ø_i > 0 or vice versa, °If Ø_{i- 1} |+|Ø_i| ≤ 180, c = |Ø_i−1|+|Ø_i

  • °If |Ø_i−1|+|Ø_i > 180, c = (180 – |Ø_i−1) + (180 – |Ø_i|).

Statistical analyses of the metrics were performed with Stata/Special Edition 13.1 (StataCorp LP, College Station, TX). We defined retention of skills (absence of skill decay) as performance within 1 standard deviation (SD) or less of baseline average for the combined cohort. For each motion metric (path length, translational motions, rotational sum, and time) and sensor (dorsum and probe), we performed the following:

1. Calculated the residents’ average and SD at the end of the course (baseline) and added 1 SD to the average to determine the threshold of retention

2. Coded each resident's performance at baseline and at the follow-up session as “1” if it met the threshold (≤1 SD of the cohort baseline average) and as “0” if it did not (>1 SD of the cohort baseline average).

3. The number of residents who met the threshold at baseline was compared with the number at follow-up using McNemar's test.

To avoid a type II error in which the presence of significant decay may not have been detected, we did not correct for the multiple comparisons made and considered a P value of <.05 to be significant.

In addition, we also performed 2 secondary analyses. First, we repeated the same analysis described above for each segment of the procedure. The procedure was segmented into 3 checkpoints, as defined previously.¹¹ Second, because time can be captured easily and does not require additional calculations as opposed to the other motion metrics, we calculated the Pearson correlation coefficient between time and each of the other metrics to provide insight into the relationship between time and the other metrics and to help determine whether time alone is an adequate marker of competence.

Results

Twelve anesthesiology residents participated in the study and were included in the analysis. Results are summarized in Table 1 and depicted in Figure 2. For context, meeting the threshold indicated less excessive motion and therefore better performance. Summary statistics (mean ± SD and median [interquartile range]) of the motion metrics are presented in Table 2.

Table 1.

Number of Residents Meeting the Threshold for Retention (N = 12)

Metric	Checkpoint	Sensor	Baseline	Follow-Up	P Value
Path length (centimeters)	Sum	Dorsum	11 (91.7%)	8 (66.7%)	.375
		Probe	11 (91.7%)	4 (33.3%)	.039a
	1	Dorsum	10 (83.3%)	6 (50%)	.219
		Probe	10 (83.3%)	5 (41.7%)	.063
	2	Dorsum	11 (91.7%)	10 (83.3%)	.999
		Probe	11 (91.7%)	8 (66.7%)	.375
	3	Dorsum	10 (83.3%)	10 (83.3%)	.999
		Probe	11 (91.7%)	7 (58.3%)	.219
Translational motions (number of motions)	Sum	Dorsum	11 (91.7%)	10 (83.3%)	.999
		Probe	11 (91.7%)	5 (41.7%)	.031a
	1	Dorsum	11 (91.7%)	8 (66.7%)	.250
		Probe	9 (75%)	5 (41.7%)	.219
	2	Dorsum	10 (83.3%)	10 (83.3%)	.999
		Probe	9 (75%)	9 (75%)	.999
	3	Dorsum	11 (91.7%)	12 (100%)	.999
		Probe	11 (91.7%)	8 (66.7%)	.375
Rotational sum (°)	Sum	Dorsum	10 (83.3%)	9 (75%)	.999
		Probe	10 (83.3%)	4 (33.3%)	.031a
	1	Dorsum	11 (91.7%)	6 (50%)	.063
		Probe	10 (83.3%)	2 (16.7%)	.008a
	2	Dorsum	10 (83.3%)	10 (83.3%)	.999
		Probe	10 (83.3%)	8 (66.7%)	.625
	3	Dorsum	10 (83.3%)	11 (91.7%)	.999
		Probe	10 (83.3%)	9 (75%)	.999
Time (seconds)	Sum	N/A	10 (83.3%)	5 (41.7%)	.125
	1	N/A	10 (83.3%)	6 (50%)	.125
	2	N/A	11 (91.7%)	8 (66.7%)	.375
	3	N/A	10 (83.3%)	7 (58.3%)	.375

Abbreviation: N/A, not applicable.

^aSignificant at α = .05.

Figure 2.

Number of residents meeting the threshold for retention (N = 12). (A) For path length, the number of residents who met the threshold at baseline was not significantly different than at follow-up for the dorsum (baseline: 11 [91.7%], follow-up: 8 [66.7%]; P = .375), but it was significantly higher than at follow-up for the probe (baseline: 11 [91.7%], follow-up: 4 [33.3%]; P = .039). (B) For translational motions, the number of residents who met the threshold at baseline was not significantly different than at follow-up for the dorsum (baseline: 11 [91.7%], follow-up: 10 [83.3%]; P > .999), but it was significantly higher than at follow-up for the probe (baseline: 11 [91.7%], follow-up: 5 [41.7%]; P = .031). (C) For rotational sum, the number of residents who met the threshold at baseline was not significantly different than at follow-up for the dorsum (baseline: 10 [83.3%], follow-up: 9 [75%]; P > .999), but it was significantly higher than at the follow-up session for the probe (baseline: 10 [83.3%], follow-up: 4 [33.3%]; P = .031). (D) For time, the number of residents who met the threshold at baseline (10 [83.3%]) was not significantly different than at follow-up (5 [41.7%]; P = .125).

Table 2.

Summary Statistics of Motion Metrics for Baseline and Follow-Up Trials (N = 12)^a

Metric	Sensor	Baseline Mean ± Standard Deviation	Follow-Up Mean ± Standard Deviation	Baseline Median (Interquartile Range)	Follow-Up Median (Interquartile Range)
Path length (centimeters)	Dorsum	1036.9 ± 174.15	971.6 ± 436.22	1073.2 (870.65 to 1154.96)	1017.8 (666.89 to 1250.67)
	Probe	470.7 ± 126.39	675.6 ± 241.54	473.7 (360.51 to 549.48)	657.2 (501.59 to 815.80)
Translational motions (number of motions)	Dorsum	199.7 ± 53.25	183.7 ± 69.91	200 (164 to 217)	187 (141.5 to 243.5)
	Probe	79.7 ± 35.83	134.4 ± 63.13	69.5 (54.5 to 98)	122 (83.5 to 183)
Rotational sum (°)	Dorsum	6472.8 ± 1606.90	5898.9 ± 3042.62	6095.3 (5171.25 to 7790.52)	6813.3 (3475.59 to 8455.75)
	Probe	4335.8 ± 1851.69	7170.0 ± 2732.09	3962.8 (3315.41 to 4319.06)	6435.6 (5146.75 to 8821.60)
Time (seconds)	N/A	95.5 ± 23.78	125.7 ± 34.92	92.1 (82.87 to 111.62)	121.6 (97.51 to 155.04)

Abbreviation: N/A, not applicable.

^aThe motion metrics presented here are for the whole central venous catheter placement trial.

Path Length

For path length, the number of residents who met the threshold at baseline was not significantly different than at follow-up for the dorsum (baseline: 11 [91.7%], follow-up: 8 [66.7%]; P = .375). By contrast, significantly more residents met the threshold at baseline than at follow-up for the probe (baseline: 11 [91.7%], follow-up: 4 [33.3%]; P = .039).

At baseline, the mean path length was 1036.9 ± 174.15 cm and 470.7 ± 126.39 cm for the dorsum and probe, respectively. At follow-up, the mean path length was 971.6 ± 436.22 cm and 675.6 ± 241.54 cm for the dorsum and probe, respectively. At baseline, the median path length was 1073.2 (870.65 to 1154.96) cm and 473.7 (360.51 to 549.48) cm for the dorsum and probe, respectively. At follow-up, the median path length was 1017.8 (666.89 to 1250.67) cm and 657.2 (501.59 to 815.80) cm for the dorsum and probe, respectively.

Translational Motions

For translational motions, the number of residents who met the threshold at baseline was not significantly different than at follow-up for the dorsum (baseline: 11 [91.7%], follow-up: 10 [83.3%]; P > .999). By contrast, significantly more residents met the threshold at baseline than at follow-up for the probe (baseline: 11 [91.7%], follow-up: 5 [41.7%]; P = .031).

At baseline, the mean number of translational motions was 199.7 ± 53.25 and 79.7 ± 35.83 for the dorsum and probe, respectively. At follow-up, the mean number of translational motions was 183.7 ± 69.91 and 134.4 ± 63.13 for the dorsum and probe, respectively. At baseline, the median number of translational motions was 200 (164 to 217) and 69.5 (54.5 to 98) for the dorsum and probe, respectively. At follow-up, the median number of translational motions was 187 (141.5 to 243.5) and 122 (83.5 to 183) for the dorsum and probe, respectively.

Rotational Sum

For rotational sum, the number of residents who met the threshold at baseline was not significantly different than at follow-up for the dorsum (baseline: 10 [83.3%], follow-up: 9 [75%]; P > .999). By contrast, significantly more residents met the threshold at baseline than at the follow-up session for the probe (baseline: 10 [83.3%], follow-up: 4 [33.3%]; P = .031).

At baseline, the mean rotational sum was 6472.8° ± 1606.90° and 4335.8° ± 1851.69° for the dorsum and probe, respectively. At follow-up, the mean rotational sum was 5898.9° ± 3042.62° and 7170.0° ± 2732.09° for the dorsum and probe, respectively. At baseline, the median rotational sum was 6095.3° (5171.25° to 7790.52°) and 3962.8° (3315.41° to 4319.06°) for the dorsum and probe, respectively. At follow-up, the median rotational sum was 6813.3° (3475.59° to 8455.75°) and 6435.6° (5146.75° to 8821.60°) for the dorsum and probe, respectively.

Time

For time, the number of residents who met the threshold at baseline (10 [83.3%]) was not significantly different than at follow-up (5 [41.7%]; P = .125).

At baseline, the mean time was 95.5 ± 23.78 seconds. At follow-up, the mean time was 125.7 ± 34.92 seconds. At baseline, the median time was 92.1 (82.87 to 111.62) seconds. At follow-up, the median time was 121.6 (97.51 to 155.04) seconds.

Secondary Analyses

The only significant difference detected in our analysis of the segments was for the rotational sum of the probe for checkpoint 1 where significantly more residents met the threshold at baseline than at follow-up (baseline: 10 [83.3%], follow-up: 2 [16.7%]; P = .008).

The Pearson correlation coefficients between time and path length, time and rotational sum, and time and translational motions were 0.31, 0.41, and 0.37, respectively.

Discussion

Motion analysis can be used to objectively assess performance in CVC placement.^9,11 Although motion analysis and alternative measures such as GRSs confirm the competency of an individual at a certain period in time, ensuring the retention of skills longitudinally is crucial for effective physician training and safe patient care. Establishing objective metrics of retention could be a hallmark advancement in resident training by identifying critical periods of decay in essential skills while reducing reliance on expert observation and increasing standardization across institutions. In this study, we observed that after 7 months without formal practice, residents exhibited skill retention in tasks involving needle insertion (their dominant hand) and skill decay in tasks involving the US probe (their nondominant hand). We also found that time had moderate/weak positive correlations with the other metrics. Residents exhibited global skill decay in tasks requiring US imaging with their nondominant hand. In scanning the neck vasculature before cannulation, an increase in path length may indicate reduced ability to identify appropriate skin landmarks and distinguish between the internal jugular vein and carotid artery.¹¹ Anecdotally, residents were observed as using excessive compression of the vessels and perpendicular sliding of the probe along the neck to orient themselves both anatomically and visually on the US screen. By contrast, more experienced clinicians would not make these excessive movements and therefore would have lower path length, as evidenced in our previous study where the average path length of the probe for experts (when performing the same procedure with the same setup) plus 1 SD was about 40% of the average path length of the probe for novices on their first trial.¹¹ The excessive compression and perpendicular sliding of the probe observed in residents may have also been due to the large footprint of the Butterfly probe, which may have contributed to difficulties with needle visualization. Novice residents may need more practice to familiarize or refamiliarize themselves with the Butterfly probe than experts.

Rotational sum has been correlated to an increase in tilting, rocking, and rotation of the probe, typically while the footprint of the probe is static.¹² Rotational sum of the US probe was significantly elevated over baseline at the 7-month follow-up. Previous studies have noted the importance of needle tracking through incremental tilting of the US probe during cannulation of the vessel, which requires dexterous handling of the US probe and visual recognition of the needle tip throughout the procedure.^16,17 Anecdotally, participants were noted during follow-up trials as failing to aspirate fluid from the vessel as a result of both perforating the vessel through and through, and missing the internal jugular vein entirely. This skill deficit may be remedied through continuous, light aspiration upon puncture of the skin and improved probe manipulation to identify the true needle tip, as opposed to the shaft. Despite being instructed during the original course and reminded at the follow-up brief review to maintain continuous light aspiration upon needle puncture, residents at the follow-up trials were observed as not performing this step, which may be another area of skill decay that contributed to less efficient hand motions. Furthermore, this skill deficit may be the result of poor imaging optimization, highlighted by the secondary analysis. By contrast, experts experienced with image optimization would have lower rotational sums when performing the same procedure with the same setup, as demonstrated in our previous study where the average rotational sum of the probe for experts plus 1 SD was about 25% of the average rotational sum of the probe for novices on their first trial.¹¹

Secondary analysis of trial data by segmentation identified rotational sum of the US probe in checkpoint 1 as a precise area of significant decay. Checkpoint 1 included the time from start to the needle's first contact with skin. Skill decay in this area may be attributable to a decreased ability to map the trajectory of the internal jugular vein through probe tilting and to center the vessel on the US screen through probe sliding and rocking before cannulation. These skills are crucial in optimizing needle trajectory, successfully cannulating the appropriate vessel, and ensuring uncomplicated guidewire placement. Given the notable skill decrement in checkpoint 1 but not others, the authors believe mastery of this subtask should be emphasized in future training.

In this study, translational motions of the US probe were significantly increased from baseline to follow-up. Previous studies have also identified a correlation between level of training and translational motions, path length, and rotational sum.¹⁵ An increase in translational motions in our current study likely represents unnecessary movements in how residents handled the US probe throughout the procedure. A reduction in translational motions could be accomplished through the use of fewer and more fluid probe motions, which may be acquired with experience. Fluid movements while handling instruments has been considered indicative of competence in a previous GRS study for CVC placement.¹⁸ This is supported by our previous study where the average number of translational motions of the probe for experts (when performing the same procedure with the same setup) plus 1 SD was about 55% of the average number of translational motions of the probe for novices on their first trial.¹¹

Analysis of the dominant hand's dorsum sensor found that the number of residents performing within 1 SD of the baseline average was not significantly different from start to the 7-month follow-up. A noninferior change in path length and translational motions implied that residents were able to successfully complete the component tasks of CVC placement without significant extraneous motion relative to baseline. In tandem with a noninferior change in their rotational sum, residents demonstrated preserved dexterity in cannulating the internal jugular vein following identification of neck vasculature by US. Regarding the metric of time, there was no significant decay in the efficiency of CVC placement. Although we did not detect any significant differences in these metrics, the number of residents who met the threshold was higher at baseline than at follow-up for all metrics for the whole procedure, which suggests a trend toward decay and support for refresher training. Follow-up measurements at a later timepoint from baseline may have resulted in detection of significant decay in the dominant hand and also time.

Although easily measured, if time alone was used to assess skill retention in this study, we would not have detected the decay we noticed in residents’ handling of the US probe. Moreover, in our sample, time did not have a strong correlation with the other metrics. These results suggest that time is not an adequate marker of motion efficiency by itself and should be combined with other motion metrics to both track competence and identify specific areas for improvement.

Limitations to this study include a relatively small sample size and the absence of motion recordings for each participant's 2 practice trials allotted at the time of 7-month follow-up, during which more decay may have been detected. Also, due to limitations of the mannikin, whose tissue would have been irreversibly damaged if it was cut and dilation was performed, analysis of hand motions throughout the whole CVC procedure, including the critical steps of fully inserting the dilator, removing the dilator, and threading the CVC itself, was not performed. Despite participants’ lack of formal CVC placement training between baseline and follow-up, they still conducted patient care in various postgraduate year 1 (intern) rotations. Therefore, they may have practiced alternative procedures involving needling and US imaging, inadvertently improving their ability to perform CVC placement. Still, given the rotations of the trainees’ intern year, the authors do not believe this occurred to a notable extent.

Future studies should perform longitudinal assessments of proficiency at regular intervals and control for potential confounders including the number and types of relevant procedures performed clinically between intervals as well as prior US training and experience. In addition, further research should include assessment of the retention of knowledge in conjunction with the retention of manual skills for a more comprehensive evaluation of performance. Moreover, further investigation into whether our results reflect a left-versus-right or dominant-versus-nondominant hand phenomenon (as opposed to whether they reflect the tasks themselves) is needed to more fully interpret our findings. Finally, as this study only assessed skills on simulated trials, future research should focus on assessing these skills when residents perform CVC placements on patients where they face challenges present in the clinical setting, such as physiologic and monitoring issues, different degrees of US clarity, and environmental distractions and pressures.

Conclusions

A comprehensive analysis of motion metrics, including path length, translational motions, rotational sum, and time, can be used to objectively assess skill decay in anesthesiology residents performing CVC placement. Significant changes in probe metrics revealed skill decay in US imaging of the neck vasculature, fluid handling of the US probe, and needle tracking. Concomitant nonsignificant changes in motion metrics of the dorsum sensor revealed skill retention in obtaining venous access, guidewire placement, and threading of the dilator to the skin. Secondary analysis by segmentation identified precannulation US scanning of the neck to be particularly sensitive to decay. This may justify more deliberate training and assessment of this segment of the procedure in future training.