The learning curve of sonographic inferior vena cava evaluation by novice medical students: the Pavia experience

Abstract

Purpose

The sonographic evaluation of inferior vena cava diameters and its collapsibility—that is also defined as the caval index—has become a popular way to easily obtain a noninvasive estimate of central venous pressure. This is generally considered an easy sonographic task to perform, and according to the American College of Emergency Physicians (ACEP) Guidelines 25 repetitions of this procedure should be sufficient to reach proficiency. However, little is known about the learning process for this sonographic technique. Therefore, we designed this study to investigate the learning curve of inferior vena cava evaluation.

Methods

We enrolled a sample of ten ultrasound-naïve medical students who received a preliminary training provided by two Junior Emergency Medicine Residents. Following training, each student performed the sonographic task on 25 different patients who were hospitalized in the internal medicine ward. The students’ performance was compared with the results obtained by a consultant in internal medicine with extensive experience in point-of-care ultrasound, who repeated the procedure on the same patients (gold standard). In detail, we evaluated the time to complete the task, the quality of the obtained images, and the ability to visually estimate and measure the caval index.

Results

Although most students (9/10) reached the pre-defined level of competence, their overall performance was inferior to the one achieved by the gold standard, with little improvement over time. However, repetition was associated with progressive shortening of the time needed to achieve readable images.

Conclusions

Overall, these findings suggest that, although allowing to obtain a pre-defined competence, 25 repetitions are not enough to reach a good level of proficiency for this technique, that needs a longer training to be achieved.

Electronic supplementary material

The online version of this article (10.1007/s40477-018-0292-7) contains supplementary material, which is available to authorized users.

Introduction

Point-of-care ultrasonography (POCUS) has gained much popularity in the last two decades, contributing to a major change in medical practice, particularly in the setting of acute care and emergency medicine. Nowadays all accredited Emergency Medicine (EM) Residency Programs in the United States provide a robust POCUS training, similarly to what occurs in European EM Residencies, though with some country-specific variations [1]. Moreover, considering the great utility and the widespread use of bedside ultrasound (US) in EM as well as in other specialties, many Medical Schools are now formally including US-based curricula also at the undergraduate teaching level. Indeed, US technology has been proven to improve learning both at the pre-clinical and the clinical stage [2, 3].

One basic task that is usually taught in all postgraduate and in most undergraduate POCUS-curricula is the sonographic visualization of Inferior Vena Cava (IVC). This technique allows a reliable noninvasive estimate of central venous pressure by combining IVC maximum diameter and its collapsibility with respiration, that is typically measured 2–3 cm before the right atrium [4].Such an estimate represents a major help in the evaluation of many different hemodynamic conditions such as shock, dehydration or hypervolemic states [5, 6], in the assessment of fluid responsiveness, as well as in completing the echocardiographic study in the setting of heart failure [7–9].

Nevertheless, the US study of IVC for a given purpose, such as the assessment of the hydration state and the prediction of fluid responsiveness in the single patient, can be poorly reliable under some circumstances. The physician should be aware of these limitations, to avoid misleading conclusions negatively influencing the subsequent clinical decisions [10]. The technique of sonographic visualization of the IVC is generally considered as relatively easy to perform, though its learning curve has been poorly studied. A recent study from Columbian investigators aimed at evaluating the learning process among EM residents who performed ultrasound on healthy volunteers with low-to-normal BMI [11]. Here we present a similar approach, assessing the learning curve for sonographic IVC visualization in a more realistic setting that included real patients with a wide range of different clinical conditions and of body phenotypes, as suggested by the ACEP guidelines [12]. We hypothesized that a real patient setting could more accurately assess the learning process of novices for this technique.

Methods

Population and operators

We enrolled on a voluntary basis ten 3-year Medical Students (seven females, three males), who were attending the University of Pavia. The only exclusion criteria consisted in having received previous theoretical and/or practical ultrasound training, during formal teaching or in the setting of elective courses. All students received a 2 h preliminary training provided by two Junior EM Residents (SDP, AB), during which they were introduced to the basic principles of ultrasonography physics. Several videos and images of most common POCUS applications were shown and discussed, with a special focus on the utility and the limitations of IVC visualization. During training, students also practiced with the hand-held sonographer, briefly scanning on each other, to familiarize with the device and learn how to acquire and save images with the correct technique and modalities as requested by the study protocol. After such a preliminary training, each student came for five consecutive days to the internal medicine ward of the San Matteo University Hospital in Pavia, where they performed five point-of-care ultrasound examinations of the IVC every day, each on a different patient, for a total number of twenty-five repetitions. A sample of 250 patients was randomly selected among those hospitalized in the internal medicine ward at the time of the study, using an online available random number generator. We excluded all patients who were either unconscious or unable to provide consent, as well as those in whom the sonographic examination would have caused pain or excessive discomfort. Immediately after the student completed each single US examination, the scan procedure was repeated by the expert attending physician, a consultant in Internal Medicine with experience in EM and EM POCUS, thereby acting as the gold standard (FF). All the collected images, both from the students and from the gold standard, were subsequently reviewed by a second consultant in Internal Medicine with certified expertise in ultrasonography (TP), who evaluated image quality using the Scale on quality assurance as proposed by ACEP [9]. All participating students and patients provided informed consent to the study.

Technique

The study was conducted using the hand-held sonographer Vscan™, GE Vingmed Ultrasound AS, Horten, Norway, provided of a phased array probe with a bandwidth of 1,7–3,8 MHz [13], that is routinely used in the Internal Medicine ward of our Hospital.

All patients were lying flat during the procedure. Operators (both the students and the gold standard) were asked to place the probe transversally in the epigastrium, then rotate it 90° until the indicator was cephalad, to obtain a longitudinal view of IVC at its entrance into the right atrium. In case the IVC could not be visualized with sub-xiphoid approach, it was possible to move to a sub-costal view by sliding the transducer along the right costal margin. To simplify and standardize the procedure, the IVC assessment through intercostal spaces was not allowed. Once operators obtained the desired image, they were asked to record a 2 s video. The procedure time was limited to a maximum of 150 s, starting from the first contact of the probe with the patient’s skin. The operators were asked to first visually estimate the caval index from the recorded video, prior to measuring the inspiratory and expiratory diameters for calculating the effective caval index according to the standard formula (caval index = IVC expiratory diameter − IVC inspiratory diameter, divided by IVC expiratory diameter × 100).

All obtained videos were reviewed and assigned an image quality grade according to the ACEP Scale, as shown in Table 1 [14].

Table 1.

Table modified from the ACEP proposed scale for the evaluation of ultrasound image quality

	1	2	3	4	5
Grading scale definitions	No recognizable structure, no objective data can be gathered	Minimally recognizable structures but insufficient for diagnosis	Minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws	Minimal criteria met for diagnosis, all structures imaged well and diagnosis easily supported	Minimal criteria met for diagnosis, all structures imaged with excellent image quality and diagnosis completely supported

According to this scale, a score 3 or greater is considered acceptable for diagnostic purposes [12]

Design

The preliminary training for the enrolled students was provided by two Junior EM Residents with limited and uncertified US expertise. The efficacy of residents’ training was assessed via a multiple choice questionnaire that was administered to the students both before and after the training session (see the attached Questionnaire). The number of repetitions per student was set at 25, which is considered by the ACEP guidelines [10] as the minimum number of repetitions that have to be performed to acquire competency in this sonographic task, as subsequently confirmed by Betancourt et al. [9].

During each bedside measurement session, the students were not in the presence of the gold standard, who was asked to wait outside the patient’s room until the procedure was completed. Once obtained the video, the students had to first visually estimate the caval index, by reporting it either as < 50 or ≥ 50%; thereafter they had to measure the inspiratory and expiratory diameter, to calculate the caval index. Only when this procedure was completed, the gold standard was asked to enter the room and to repeat the procedure, the students being allowed to attend the expert US evaluation.

Patient’s variables included only estimated BMI that was classified into three categories (underweight to normal, overweight to class 1 obesity, class 2 obesity or above). The variables considered for the operators included: time to complete each repetition (s), intra-operator and inter-operator correspondence between visual estimated and measured caval index (dichotomized as < 50 or ≥ 50%), image quality. Moreover, a pre-set level of competency for the procedure was defined as the ability to obtain four consecutive sufficient images (ACEP Score ≥ 3), which corresponds to a 80% success rate over five consecutive repetitions.

This study was conducted between February and June 2017. All data obtained, including videos for subsequent review, were anonymized and electronically stored for subsequent analyses.

Statistical analysis

Descriptive statistics were produced for all variables. Mean and standard deviation (SD) are presented for normally distributed variables, and median and interquartile range (IQR) for non normally distributed variables, number and percentages for categorical variables. Groups were compared with Fisher exact test for categorical variables. Tests for paired data were applied whenever relevant. Cohen kappa coefficient was used to assess concordance between gold standard and students. In all cases, 2-tailed tests were used. P value significance cut-off was 0.05. The effect of learning was explored by comparing the first 13 procedures to the remaining 12.

Results

The gold standard obtained an US-image in 228 patients out of the 250 examined (92%), a rate that is in line with other previous similar studies [13, 14]. One hundred forty-nine images (59.6%) were considered qualitatively valid by the external reviewer. The most common causes of poor quality images were severe obesity, meteorism, poor patient’s compliance, ascites as well as barrel chest or tortuous shaped IVC.

The total time he needed to perform the repetition ranged from 7 to 150 s (median 35 s, IQR 21–73.5 s). In addition, gold standard’s visual estimation of caval index corresponded to the measured values in 85% of cases in whom it was possible to obtain valid images (kappa (κ) 0.69, P value < 0.001).

All students significantly increased their number of correct answers on the questionnaire administered before and after the preliminary training (Fig. 1).

Fig. 1.

Graphic representation of student’s correct answers to questionnaire before and after the preliminary training Students were overall able to obtain an image in 187 out of 250 cases (75%, P < 0.001 when compared with the gold standard). Image quality was deemed as sufficient (ACEP ≥ 3) in 116/250 cases (46.4%)

Nine out of 10 (90%) students reached the pre-defined level of competency (Fig. 2).

Fig. 2.

Gradual attainment of the level of competency over the 25 repetitions. Within the first 5 iterations none of the students reached the pre-set level competency, which was, however, accomplished by 90% of students by the end of the study

On average, students needed a median time of 81 s to complete the task (IQR 45–128, P < 0.001 compared with gold standard). Such a procedure time improved with the progression of training (Fig. 3), from 88 s (IQR 59–130) in the first 13 procedures versus 67 s (IQR 34–120) in the remaining 12 (P = 0.004).

Fig. 3.

Median time needed by students to complete the task during the first 13 and the last 12 iterations

Both students and gold standard were more likely to obtain low quality images (or even fail) in patients with higher estimated BMI (Table 2).

Table 2.

Percentage of success in obtaining a valid (ACEP score ≥ 3) US image in different body types (BMI class 0 = underweight or normal weight, BMI 1 overweight or class 1 obesity, BMI 2 Class 2 obesity or above)

	BMI class
	0 (%)	1 (%)	2 (%)
Valid image Gold Standard n = 250	68	62	49
Valid image Student n = 250	50	48	36

Concordance between student’s visually estimated caval index and their measured caval index occurred in 138 out of 187 cases (74%), (κ 0.44, P < 0.001), with a non-significant improvement over time (72% in the first 13 repetitions and 76% in last 12; P = 0.637).

A correspondence between the visual estimation of caval index provided by the students and the gold standard’s one was observed in 120 out of 185 cases (65%) (κ 0.28, P = 0.001). Also in this case, no change was observed over the course of the study (correspondence being 66.3% in the first 13 attempts and 63.3% in the last 12, respectively; P = 0.79).

Similarly, the correspondence between the measured caval index values was observed in 131 out of 185 cases (71%) (κ 0.33, P = 0.001), without any significant improvement throughout the study (70.8% in the first 13 attempts and 72.6% in the last 12; P = 0.69).

Discussion

The introduction of US training in undergraduate medical programs represents a relatively recent acquisition. In the last years a growing number of Medical Schools have developed a variety of US courses specifically addressing medical students, to teach basic sonographic skills and competencies that they will need later on in their medical career [2, 3, 15]. Interestingly, many of these competencies are directly derived from POCUS Curricula specifically designed for EM Residents and Physicians, and indeed many undergraduate US as well as European Curricula do include rotations in the Emergency Department or Critical Care Unit [15].

At our institution, undergraduates are not systematically trained in ultrasound during the 6 year Medicine course, although several students are occasionally exposed to ultrasound evaluation during their clinical rotations or elective courses.

Since aim of the present study was the assessment of the learning curve for a given sonographic task, a convenience sample of ultrasound-naïve students seemed the most appropriate target group. In particular, we addressed 3rd year medical students who had just completed their Medical Semeiotics course, during which they learned the fundamentals of physical examination: this was done to better instill in our students the vision of POCUS as a reasonable extension and integration of the traditional physical examination, rather than an alternative.

For the conduction of the study, the internal medicine ward appeared to be the optimal setting, in fact, this allowed students to encounter patients with a wide range of different clinical conditions, including cases of septic and cardiogenic shock, severe tricuspid insufficiency, pulmonary embolism, heart failure, renal failure, etc., thus resulting in a wide diversity of inferior vena cava appearance. At the same time, such an approach guarantees a much quieter and less crowded environment as compared to the Emergency Department, making bedside teaching easier and more comfortable.

The decision to use a hand-held sonographer was due to the quick and unlimited availability of such a device in the clinical ward, to its simplicity of use and its handiness. This kind of device has some advantages on more complex ultrasound systems, as it can fit in a pocket and it can lit up in few seconds. For these practical reasons it is often used in emergency and bedside quick assessment, despite having some technical limitations, namely in image definition and color-Doppler analysis. Nevertheless, the hand-held sonographer used in the present study was validated for several uses, including adult and pediatric cardiac evaluation [16], and has been successfully applied in a variety of clinical settings [17–19].

The preliminary training offered to the students before using the US technique at the bedside was provided by two Junior EM Residents (SDP and AB). Its efficacy was proved by the students’ response to a questionnaire that was specifically designed to assess the quality of training. Moreover, most students (9/10) were able to produce at least one good quality image already in the first measurement session. This indirectly underscores that EM Residents were able to transmit not only the theoretical basis but also the practical skills that are needed for achieving this task. As predictable, students obtained a lower number of images than the gold standard (75 vs. 92% of all patients). However, the quality of the obtained images was almost comparable to the gold standard (63 vs. 65.4% of the obtained images).

Nine out of ten students reached the level of competency that was pre-set at 4 consecutive valid images: it has to be noted that a single image with ACEP Score < 3 made the repetition to be discarded, interrupting the positive sequence of the student, only if the gold standard was able to obtain a sufficient quality image for that given repetition.

The average time that the students needed to perform each repetition improved over the course of the study, decreasing from 90.9 s in the first thirteen repetitions to 75.1 s in the last twelve. In comparison, the gold standard completed the task in a much shorter time (average time 52.9 s), although this was more than double when compared with the time reported by Betancourt et al. (17.3 s) [9]. It has to be kept in mind that the latter study was performed on healthy volunteers with low-to-normal BMI [9], whereas the present investigation was conducted in a more realistic setting, such as the internal medicine ward, that included patients with a wide range of different clinical conditions and of body phenotypes, as suggested by the ACEP guidelines [10].We also analyzed the ability of the students to visually estimate the caval index, as < 50% or ≥ 50%, as compared to their measured value for a given repetition (intra-operator correspondence): this parameter did not show a significant improvement over time, and kappa value was only moderate (0.44). However, students succeeded in most cases showing a performance slightly inferior to that of the gold standard (74 vs. 85% of cases).

In contrast, when we analyzed the correspondence of students visually estimated and measured caval index with the values obtained by the gold standard (inter-operator correspondence), there was a match in, respectively, 65 and 70.8% of cases, without any significant improvement over time and low-to-moderate kappa values (0.28 and 0.33, respectively).

The patients that were randomly enrolled in the study were far from being the ideal model for sonography: as already described, many of them had either a “difficult” body habitus or comorbidities that made the performance of the sonographic task quite challenging. In particular, we were able to demonstrate a significant inverse correlation between the chance of success and the presence of obesity, especially for obesity classes higher than II (BMI > 35). Indeed, this represents a real strength of our study that allowed a realistic assessment of the learning process for the sonographic visualization of inferior vena cava and the estimation of the caval index in a “realworld” setting.

In our study, the 25 repetitions suggested by ACEP guidelines appear to be sufficient to acquire only a limited proficiency in caval index evaluation, without reaching the skill of a trained clinician who is used to perform the technique since a long time in his clinical practice.

Limitations

The study presents several limitations. First of all, even though the number of novices that we enrolled was greater compared to the study of Betancourt at al [9], a greater group of participants would have strengthened the key message of the study. The sonographic assessment of inferior vena cava diameters and the calculation of caval index have some intrinsic limitations that can affect the accuracy of measurements and the inter-rater reliability [20], the most important ones probably being the variation of patient’s respiratory rate and the volume status.

We tried to overcome the latter by reducing the temporal proximity between student’s and gold standard’s measurements that in all cases was limited to few minutes, thus making a sonographic detectable variation of volume status unlikely.

Compared to most standard sonographers, the hand-held device that we used has a smaller screen, a lower image resolution and it lacks M-mode, although recent finding suggest that B-Mode is superior to M-Mode in providing more reliable measurements of IVC diameters [21]. Moreover, to obtain caliper measurements on the Vscan™, operators had to freeze the obtained videoclip at the point of maximum and minimum IVC diameter; both students and the gold standard found this quite uncomfortable as in some cases it was not so easy stopping the video at the right frame. Moreover, an intrinsic limitation of the device is that the clips last only 2 s, making sometimes difficult to record a full-inspiratory and full-expiratory diameter in a single videoclip. We tried to overcome this limitation by immediately repeating some of the clips when the operator had the impression of incomplete recording of the respiratory cycle.

Eventually, while each student performed 25 repetitions on 25 different patients, in a limited number of cases the gold standard repeated the exam on the same patient he already scanned previously, thus with a potential bias for these measurements.

Conclusions

The sonographic visualization of inferior vena cava and the evaluation of caval index confirmed to be relatively easy tasks to learn. Novices rapidly acquire the capacity to obtain valid images, to visually estimate and measure the caval index. However, 25 repetitions, which is the number of practical iterations suggested by ACEP to provide adequate ultrasound training, seem insufficient to achieve the highest level of competency for this technique. Nevertheless, it is very likely that the skills acquired in 25 measurements are sufficient for further self-training without further supervision.

The technical skills required to perform this sonographic task as well as the theoretical knowledge that lies behind the technique, can be easily transferred by Junior Emergency Medicine Residents with basic experience in POCUS.

Further studies are needed to assess the capacity of novices to not only learn this technique, but also to interpret the obtained results and to integrate them in the clinical decision-making process.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Compliance with ethical standards

Conflict of interest

We declare no conflict of interest or financial support for this article.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.