Performance of a 25% Inferior Vena Cava Collapsibility in Detecting Fluid Responsiveness When Assessed by Novice Versus Expert Physician Sonologists

Abstract

Objectives:

Inferior vena cava collapsibility (cIVC) measured by point-of-care ultrasound (POCUS) has been proposed as a non-invasive means of assessing fluid responsiveness. We aimed to prospectively evaluate the performance of a 25% cIVC cutoff value to detect fluid responsiveness among spontaneously breathing intensive care unit (ICU) patients when assessed with POCUS by novice versus expert physician sonologists.

Methods:

Prospective observational study of spontaneously breathing ICU patients. Fluid responsiveness was defined as a ≥ 10% increase in cardiac index following a 500 ml fluid bolus, measured by bioreactance. Novice sonologist measured cIVC with POCUS. Their measurements were later compared to an expert physician sonologist who independently reviewed the POCUS images and assessed cIVCs.

Results:

Of the 85 participants, 44 (52%) were fluid responders. A 25% cIVC cutoff value performed better when assessed by expert sonologists than novice physician sonologists (ROC=0.82 [0.74–0.88] versus ROC=0.69 [0.60=0.77]).

Conclusions:

A 25% cIVC cutoff value measured by POCUS detects fluid responsiveness. However, the experience of physician sonologist affects test performance and should be considered when interpreting and clinically using cIVC to direct IV fluid resuscitation.

Introduction

There is growing concern^1–3 and mounting evidence^4–7 that excess intravenous (IV) fluid resuscitation is harmful to patients. Because only half of critically-ill patients will respond to IV fluids by increasing organ perfusion,⁸ a large number of patients are given IV fluids without benefit. Point-of-care ultrasound (POCUS) is widely available in both resource-rich and poor environments.⁹ It has the potential to enable clinicians to tailor fluid resuscitation to patients’ needs, ^10,11 including withholding IV fluids when they might be detrimental. POCUS is commonly used in the emergency department and intensive care unit (ICU) to direct fluid resuscitation,¹² but this practice is informed by incomplete evidence.

Measurement of inferior vena cava collapsibility (cIVC) by POCUS has been evaluated as a non-invasive means to detect fluid responsiveness in spontaneously breathing patients.^13–18 However, a lack of sufficient evidence in this patient population prevented the Society of Critical Care Medicine 2016 Ultrasound Guidelines from making a consensus recommendation on the optimal cIVC cutoff value that reliably distinguishes fluid responsiveness from nonresponsiveness.¹⁹ A subsequent meta-analysis notes significant heterogenity across studies and suggests that further research is needed before the use of cIVC is widely adopted.²⁰ Proposed cIVC cutoff values to detect fluid responsiveness range from 15 to 50%.^11,13–18 A 2014 study of mechanically ventilated patients which combined limited cardiac echocardiography with assessment of cIVC found that POCUS directed resuscitation reduced the amount of IV fluid administered to critically-ill patients, and was associated with reduced mortality.²¹ Unfortunately, these encouraging results have not been replicated among spontaneously breathing patients. A 2018 randomized control trial found no benefit to using a POCUS resuscitation protocol in hypotensive patients. However, this study used a high cIVC cutoff value (50%) to detect fluid responsiveness and did not reduce the amount of IV fluid administered during ultrasound guided resuscitation.²² If clinicians intend to use cIVC to tailor IV fluid resuscitation practices, appropriate cIVC cutoff values must be identified and confirmed, with consideration of usual POCUS practice conditions.

In 2017, we derived and proposed a cIVC cutoff value of 25% to detect fluid responsiveness for spontaneously breathing, supine, ICU patients taking non-standardized breaths.¹⁸ We compared our proposed cIVC cutoff value to distinguish fluid responders from non-responders against previously suggested values. We concluded that a 25% cIVC cutoff value was more accurate than other proposed values and produced the fewest misclassifications. Our derivation study relied on experienced physician sonologists to obtain POCUS images and assess cIVC. In usual practice, however, physicians range from being novice to expert sonologists. Data on the interrater agreement in cIVC measurement between experienced and novice sonologists is mixed, with some studies showing poor to moderate agreement,^23–25 while others report good agreement among physicians with resident-level ultrasound training.²⁶ In training programs, a novice physician sonographer usually performs the POCUS. An expert physician sonographer either reviews the images simultaneously with the novice (potentially affecting the image quality and the cIVC assessment) or reviews them at later time (“overreads” them for quality improvement or billing). It is therefore necessary to evaluate cIVC cutoff value performance under these typical conditions.

Our primary aim was to evaluate the performance of a cIVC 25% cutoff when assessed by novice physician sonologists who obtained POCUS images of cIVC in the ICU, compared to an expert physician sonologist who independently assessed cIVC during a review of ultrasound video clips obtained by the novice physician sonologists. Our secondary aim was to assess if a range of cIVC values, rather than a single cutoff, might perform better in determining fluid responsiveness.

Methods

Study Participants, Setting, and Novice Physician Sonologists

We conducted a prospective observational investigation from November 2016 to July 2018 in two medical ICUs in the United States. Patients were eligible for participation if they were breathing spontaneously, and had signs of acute circulatory failure at any time within 72 hours of presentation to the emergency department. Details of the enrollment criteria have been described previously and are outlined in the appendix.¹⁸ Informed consent was obtained from the patient or their surrogate prior to study enrollment. The novice physician sonologists (n=5) were four critical care medicine fellows and an internal medicine ICU hospitalist. One of the critical care fellows had received ultrasound training during their emergency medicine residency, the others had no prior formalized training. Each underwent a 3-hour mandatory training session to demonstrate their ability to obtain the required cIVC measurements reliably on 10 participants before obtaining the POCUS cIVC measurements for this study. The expert physician sonologist was an attending ICU physician who has performed and reviewed over 250 cIVC ultrasound measurements. The institutional review board of the participating hospitals approved the study protocol.

Study protocol and Measurements

Following enrollment, the participant’s hemodynamic profile was recorded at one-minute intervals using bioreactance (NICOM™, Cheetah Medical, Tel Aviv, Israel). The novice physician sonologists were blinded to the NICOM results. The participant first was placed in a supine position for a 3-minute NICOM calibration period. Afterwards, the novice physician sonologists recorded two consecutive, 10-second, ultrasound video clips of the participant’s IVC using a Sonosite Edge (Bothell, WA) in 2D B-mode. These video clips were uploaded to a secure server but not reviewed by the novice physician sonologist. Next, the novice performed POCUS of the IVC twice at the bedside using video capture and immediate review. They selected a still image from the videos using the cineloop function to review the respiratory cycle and obtain contemporaneous measurements of the maximum and minimum IVC diameter, 3cm caudal to the cavoatrial junction.²⁷ Visualization of the hepatic vein(s) and/or the cavoatrial junction were used to differentiate the IVC from the aorta. The participants’ breaths were not standardized during this process. After the measurements were obtained, the participant was administered a 500 ml normal saline bolus using a pressure bag through their largest gauge IV. Fluid responsiveness as measured by NICOM was defined as a ≥ 10% increase in cardiac index following the fluid bolus.²⁸

At a later date, an expert physician sonologist conducted a video review of the two consecutive, 10-second, ultrasound video clips obtained by the novice physician sonologist. Ultrasound images were reviewed using OsiriX Imaging Software (©Pixmeo, Switzerland). The expert physician sonologist was blinded to both the novice physician sonologist cIVC assessments and the NICOM results. During this review, the expert physician sonologist identified the maximum and minimum IVC diameters by visual inspection. The expert physician sonologist froze the video images at the maximum and minimum IVC diameters then formally measured these diameters at 3 cm caudal to the cavoatrial junction using calipers. cIVC (or “caval index”) was defined as: (IVC expiratory diameter – IVC inspiratory diameter)/IVC expiratory diameter.¹⁸ If the first 10-second video clip was missing or had insufficient image quality, the second video clip was reviewed instead.

To test the importance of the experience of the expert physician sonologist, the cIVC video review was independently repeated by a junior reader. The junior reader was one of the five novice sonologists with no prior experience reviewing cIVC images, was trained during a 3-hour review session, and required to show competence with 10 proctored video reviews with the expert physician sonologist.

Statistical Analysis

Enrollment, participant demographic and clinical characteristics, and participant clinical outcomes were summarized. Fisher’s exact and Wilcoxon rank-sum tests were used to compare fluid responders versus non-responders for dichotomous and continuous participant characteristics and outcomes, respectively. The performance characteristics (area under the receiver operator characteristic curve (ROC), sensitivity, specificity, predictive values, and likelihood ratios) of a 25% cIVC cutoff value in distinguishing fluid responsiveness were calculated. Performance characteristics were calculated for the cIVCs assessed by the novice physician sonologists and separately by the expert physician sonologist. The Wilson method was used calculate 95% confidence intervals (CIs) for the performance characteristics.

Lower and higher ranges of cIVC cutoff values that fit the collected data were identified separately for the novice and expert physician sonologists. The lower range cIVC cutoff value was identified by setting the sensitivity to 90%, and the higher range identified by setting the specificity to 90%. The remainder of the performance characteristics (ROC, predictive value, likelihood ratios) were calculated using the identified cIVC cutoff values. Plots of the probability of participants of being a fluid responder along with corresponding 95% CIs were created using the cIVC cutoff range values for the novice physicians and the expert physician sonologist.

Kappa and intraclass correlation coefficient (ICC) for absolute agreement using one-way random-effects linear regression and two-way random-effects linear regression were calculated for the baseline cIVC measurements to determine intra- and inter-rater reliability between the expert and junior’s first ultrasound read.

Results

Participants and fluid administration

We assessed 494 ICU patients to enroll 145 as participants. Of the 145 enrolled participants, 13 withdrew before study initiation, 12 were excluded due to NICOM error, in 21 the novice physician sonologist was unable to visualize the IVC, and in 14 the IVC video image was not adequate for review by the expert physician sonologist, yielding a final study sample of 85 participants (Figure 1). Participant characteristics are described in Table 1. Forty-four participants (52%) were fluid responders. The majority of the study participants suffered from sepsis, septic shock, diabetic ketoacidosis, or the hyperosmolar hyperglycemic state. No statistical differences were found between fluid responders and non-responders except that fluid responders were likely to have an initial higher mean arterial pressure and non-responders were more likely to have pulmonary hypertension. The median amount of IV fluid administered prior to study enrollment for all participants was 3000 ml [IQR: 2000–4000]. The median amount of IV fluid given during the study was 500 ml [IQR: 500–500] and boluses were administered over a median of 8 minutes [IQR: 6–10].

Figure #1:

Enrollment diagram

Table #1.

Participant Demographic and Clinical Characteristics and Clinical Outcomes, Fluid Responders vs. Non-responders

Patient characteristics	Fluid Responders (n = 44)	Fluid Non- Responders (n = 41)	P valuea p<
Demographic and clinical characteristics
Age (years), median (IQR)	52 (34–66)	48 (29–64)	0.32
Male gender, n (%)	27 (61.4)	27 (65.9)	0.82
BMI (kg/m2), median (IQR)	26.6 (23.0–31.5)	25.4 (21.9–28.3)	0.19
APACHE II (score), median (IQR)	14 (10–16)	16 (10–20)	0.22
Mean arterial pressure (MAP, mmHg)	74 (61–110)	64 (55–75)	0.01
Heart rate (HR)	104 (94–115)	110 (96–119)	0.45
Respiratory Rate (RR)	28 (24–34)	25 (22–30)	0.15
Fluid and other resuscitation
IVF prior to ultrasound (ml), median (IQR)	3,000 (2,000–4,000)	3,000 (2,000–5,000)	0.55
IVF administered during study (ml), median (IQR)	500 (500–500)	500 (500–500)	0.33
Duration of fluid bolus (min), median (IQR)	8 (6–11)	9 (7–10)	0.47
Required vasopressor, n (%)	4 (9.1)	8 (19.5)	0.22
Medical history, n (%)
Hypertension	23 (52.7)	21 (51.2)	> 0.99
Diabetes mellitus	24 (54.6)	23 (56.1)	> 0.99
Cardiomyopathy	4 (9.1)	5 (12.2)	0.73
Alcohol Abuse	11 (25.0)	11 (26.8)	> 0.99
COPD	10 (22.7)	6 (14.6)	0.41
Pulmonary embolism	2 (4.6)	2 (4.9)	> 0.99
Pulmonary hypertension	0 (0.0)	6 (14.6)	0.01
Hospital discharge diagnosis, n (%)
Sepsis/septic shock	9 (20.5)	16 (39.2)	0.10
DKA/HHS	16 (36.7)	16 (39.2)	0.83
Gastrointestinal hemorrhage	4 (9.1)	4 (9.8)	> 0.99
Alcohol withdrawal	1 (2.8	1 (2.4)	> 0.99
Respiratory failure (pneumonia)	2 (4.6)	1 (2.4)	> 0.99
Outcomes
ICU length of stay (days)	2 (1–2)	2 (1–3)	0.87
Hospital length of stay (days)	3.5 (2–6)	4 (3–8)	0.14
Alive at discharge	41 (93.2)	39 (95.1)	> 0.99

APACHE II = Acute Physiology and Chronic Health Evaluation II

BMI = body mass index, COPD = chronic obstructive pulmonary disease, DKA = diabetic ketoacidosis, HHS = hyperosmolar hyperglycemic state, IQR = interquartile range, ml = milliliters, n = number

Performance of the 25% cIVC cutoff value: novice vs. expert physician sonologists

The median cIVC observed for fluid responders was 38.2% [IQR: 31.5–49.9] compared to 12.9% [IQR: 7.8–21.8] for non-responders; p<0.001. The cIVC cutoff value of 25% performed better when assessed by the expert sonologist compared to the novice sonologist (ROC=0.82 [0.74–0.88] vs. ROC=0.69 [0.60–0.77]; p=0.006) (Table 2 and Figure 2). The 25% cIVC cutoff value showed substantial variations in performance across novice physician sonologists, whose ROCs for fluid responsiveness determination ranged from 0.65 to 0.85 (Supplementary Table #1).

Table #2.

Test performance characteristics of 25% cIVC cutoff value versus alternative cIVC cutoff value range by the novice physician sonologists and the expert physician sonologist

cIVC cutoff value method	Sonologist	cIVC %	ROC [95% CI]	Sensitivity % [95% CI]	Specificity % [95% CI]	PPV % [95% CI]	NPV % [95% CI]	LR+ [95% CI]	LR- [95% CI]
25% cIVC	Novice physician sonologists	≥ 25	0.69 [0.60–0.77]	70 [55–81]	68 [53–80]	70 [55–81]	68.3 [53–80]	2.2 [2.1–2.3]	0.44 [0.41–0.47]
	Expert physician sonologist	≥ 25	0.82 [0.74–0.88]	86 [73–94]	78 [63–88]	81 [68–90]	84 [70–93	3.9 [3.6–4.2]	0.17 [0.16–0.20]
Alternative cIVC cutoff value range	Novice physician sonologists	≥ 15	0.63 [0.54–0.72]	90		60 [48–71]	78.9 (56.7–91.5)	1.4 [1.4–1.5]	0.25 [0.22–0.29]
		≤ 40	0.60 [0.51–0.69]		90	77 [53–91]	55 [43–67]	3.1 [2.8–3.5]	0.77 [0.75–0.79]
	Expert physician sonologist	≤ 22	0.83 [0.75–0.91]	90		80 [67–89]	87 [74–96]	3.7 [3.5–4.0]	0.12 [0.10–0.14]
		≥ 37	0.72 [0.63–0.80]		90	86 [69–94]	65 [52–76]	5.6 [4.9–6.4]	0.50 [0.48–0.53]

cIVC = inferior vena cava collapsibility, LR = likelihood ratio, NPV = negative predictive value, PPV = positive predictive value, ROC = receiver operating curve

Figure #2:

Receiver operating curves for 25% cIVC cutoff value for the novice physician sonologists as compared to the expert physician sonologist. The left panel displays ROCs using previously derived a priori cIVC=25%, the right displays ROCs for the continuous study data.

During video review the expert physician sonologist and junior reader demonstrated moderate inter-rater reliability with a kappa = 0.52 and an intraclass correlation (ICC) = 0.73. Using the cIVC cutoff of 25% the junior video reviewer’s reads produced a ROC = 0.69 [0.55–0.79]. There was excellent intra-rater reliability for the expert sonologist between the first and second ultrasound videos; kappa = 0.96 and ICC = 0.95.

Performance of the cIVC cutoff value range approach

Table 2 provides the performance characteristics from using the cIVC cutoff value range approach, for which the cIVC cutoff value range was identified by fixing the sensitivity and then the specificity at 90%. The cIVC cutoff value range for the novice physician sonologists (15–40%) was wider than for the expert physician sonologist (22–37%). The performance characteristics of the cIVC cutoff value range were better for the expert physician sonologist than the novice physician sonologists. For the cIVC cutoff value range identified, the probability of being a fluid responder was less precise (i.e., range of probabilities was wider) for the novice physician sonologists (38.2 to 63.6%) than the expert physician sonologist (41.7 to 60.8%) (Figure 3). In other words, there was less accuracy and lower assurance in detecting fluid responsiveness when the cIVC cutoff value range was assessed and applied by the novice physician sonologists than for the range assessed and applied by the expert physician sonologist.

Figure #3:

Probability of a participant being a fluid responder using the cIVC cutoff value ranges, novice physician sonologists versus expert physician sonologist. The gray areas indicate the 95% confidence intervals for probability of being a fluid responder. The vertical red lines demarcate the cIVC cutoff range values. Circles represent individual participants who are fluid responders (top of graph) or non-responders (bottom of graph).

Discussion

In 2017 we derived and proposed a cIVC value of 25% to detect fluid responsiveness among supine spontaneously breathing ICU patients.¹⁸ Our current study prospectively tested this cIVC cutoff in the hands of novice and expert sonologists and further supports its use as the best single cIVC cutoff value, however cIVC performance depends upon the experience of the physician sonographer. As demonstrated in this study, although a 25% cIVC cutoff value to determine fluid responsiveness performed well when measured by an expert physician sonologist, its performance was diminished when measured by novice physician sonologists (ROC=0.82 [0.74–0.88] vs. ROC=0.69 [0.60–0.77]).

Several reasons might explain why expert physician sonologists performed better than novices at measuring cIVC. First, the expert physician sonologist used a high quality and large video display screen when reviewing the ultrasound images and obtaining the cIVC measurements. The novice physician sonographers instead used the smaller and lower quality display screen of the point-of-care ultrasound machine (Sonosite Edge). Second, the novice physician sonologists may have felt pressure to complete their assessments quickly while in the presence of the study participant. In contrast, the expert physician sonologist completed assessments privately and without a perceived time pressure. While both the expert and novice sonologists could scroll back and forth through the video loops to maximize/minimize IVC images, anecdotally the novices did this less. Third, the 25% cIVC value cutoff performance varied greatly across novice physician sonologists (ROC: 0.65–0.85), which suggests differences in POCUS and cIVC measurement abilities (Supplementary Table #1). This highlights how skill can vary significantly between physician sonologists and must be considered when using cIVC clinically to direct IV fluid resuscitation. Fourth, our observation that the junior video reviewer produced similar results during video review (ROC=0.69) compared to the novice physician sonologist point-of-care measurements (ROC=0.69), suggests that reader experience was a significant factor between expert’s and novices’ measurements. As POCUS becomes more widely used by internal medicine trained physicians, residency programs should adopt standardized training curricula in order to mitigate these potential sources of error in using cIVC to direct IV fluid resuscitation.²⁹

In response to concerns about sonologist skill variation, Belmont et al. tested an automated computer algorithm that can continuously measure cIVC during POCUS.³⁰ Although sonologists must properly position the ultrasound probe to obtain quality IVC images, the computer algorithm eliminates the need for an experienced reader to interpret these images. If this computer algorithm is validated, it has the potential to improve the ability of novice sonographers to obtain cIVC measurements and expand the use of cIVC to assist in determining fluid responsiveness by eliminating reader error. In the interim, our results indicate greater training is needed in assessing cIVC by POCUS than was provided to the novice physician sonologists (3 hours; 10 POCUS cIVCs) in this investigation. The optimal components and length of that training are not known, but should be the subject of future research.

Effective application of cIVC to guide IV fluid resuscitation requires clinical context. Recent research using artificial intelligence suggests that many physicians administer IV fluids past an inflection point where further fluids cause patient harm, and that vasopressors should be utilized earlier during resuscitation.³¹ While we prospectively tested a single cIVC cutoff value of 25% for expert and novice physician sonologists, we also examined our data using a gray zone approach. This produced a range of 22–37% for experts, and 15–40% for novices (Table #2, Figure #3). Considering the growing evidence that increased amounts of IV fluid harm patients, we propose a resuscitation strategy that uses a practical gray zone of 25–40% (Figure #4). When applied to our data, only 27% of study participants fall within our proposed gray zone. Therefore in 73% of cases the provider has a high level of certainty of fluid responder status. Using our proposed gray zone and a conservative approach to fluid resuscitation, we advise administering fluid to patients whose IVC collapses >40%. A more moderate approach would advise administering IV fluid to patients with a cIVC >25%. While this threshold may appear low, if employed it would have prevented IV fluid from being administered to 45% of participants in this study. Finally, we advise against adopting a liberal resuscitation strategy, unless the clinical context has unique circumstances that would call for high-volume IV fluid resuscitation.

Figure #4:

IV fluid resuscitation strategies based upon cIVC. The gray shading represents our practical proposed gray zone of cIVC 25 to 40%.

Our study has limitations. First, we performed all ultrasounds with patients in the supine position, while other authors have examined patients in the semi-upright position. Placing a patient supine might increase venous return to the right heart and consequently affect cIVC measurement, however this phenomenon has not been observed in prior research.³² Second, our study did not control for respiratory effort or intra-abominal pressure, which is known to effect cIVC.³³ Preau et al. demonstrated that patients able to take standardized breaths had greater IVC collapse.¹⁶ Notably, 25% of their participants were unable to take a standardized breath, excluding them from analysis. In contrast, our study population was comprised of patients taking non-standardized heterogenous breaths. While potentially leading to variability in cIVC assessments our heterogenous patient population increases external validity. Third, a large number of patients who were screened for the study were excluded because they either had a contraindication to IV fluid, were too ill to provide consent, or declined participation. This may limit study generalizability. In addition, a significant portion (24%) of the ultrasounds obtained by the novice physician sonologists were not usable, either due to their inability to capture the image at bedside (14%) or poor quality of the videos (10%). This proportion is higher than our previous work (12%) during which images were obtained by an experienced sonologist,¹⁸ yet likely reflects real world clinical practice and is similar to the rate observed (22%) in a large multicenter study of intubated ICU patients.³⁴ Fourth, novice physician sonologists and not the expert physician sonologist performed POCUS. We purposely designed our study to mimic typical ultrasound practice in training programs. In clinical practice, the novice physician sonographer usually performs POCUS and the expert physician sonographer either reviews the images simultaneously with the novice or reviews them at later time, as was done in this study. This approach permitted us to compare cIVC assessment between expert and novice physician sonologists when the expert was not present to affect the novice’s performance. Fifth, we did not have multiple expert sonologists read the cIVC images to assess expert inter-rater reliability. It is possible that skill may vary between experts and have affected cIVC performance. Sixth, there is debate over the accuracy of the NICOM non-invasive hemodynamic profile compared to thermodilution and bioreactance is not widely accepted in the intensive care community.³⁵ Some experts have expressed concerns that bioreactance performs better in operative patients than medical ICU patients, however convincing data supporting this assertion is lacking.³⁶ Nevertheless, multiple trials have shown good correlation between NICOM and hemodynamics measured by a pulmonary artery catheter.^37,38 To date no other form of non-invasive hemodynamic monitoring has demonstrated clinical superiority.³⁹ While recognizing the potential for bias in the NICOM device, we believe that NICOM is an appropriate and reasonable non-invasive method of measuring fluid responsiveness.

Conclusion

Our data support the use of a 25% cIVC cutoff value to detect fluid responsiveness among spontaneously breathing ICU patients, however physicians must factor in sonologist experience when using cIVC to make clinical decisions. POCUS cIVC is better able to predict fluid responsiveness when performed by an expert compared to a novice physician sonologist. Based on our research, a practical cIVC cutoff range of 25–40% in combination with clinical judgement is a reasonable alternative to a single cIVC cutoff value and might overcome variations in patient and clinical characteristics as well as sonologist skills.