Image Segmentation and Machine Learning for Detection of Abdominal Free Fluid in Focused Assessment With Sonography for Trauma Examinations: A Pilot Study

Abstract

The objective of this pilot study was to test the feasibility of automating the detection of abdominal free fluid in focused assessment with sonography for trauma (FAST) examinations. Perihepatic views from 10 FAST examinations with positive results and 10 FAST examinations with negative results were used. The sensitivity and specificity compared to manual classification by trained physicians was evaluated. The sensitivity and specificity (95% confidence interval) were 100% (69.2%–100%) and 90.0%(55.5%–99.8%), respectively. These findings suggest that computerized detection of free fluid on abdominal ultrasound images may be sensitive and specific enough to aid clinicians in their interpretation of a FAST examination.

The use of point-of-care ultrasound (US) in the setting of trauma has become the standard of care for rapid detection of abdominal free fluid in emergency medicine and trauma critical care. Free fluid in the right upper quadrant on a focused assessment with sonography for trauma (FAST) examination has been shown to be a strong independent predictor of the need for therapeutic laparotomy in trauma.^1–3 Timely detection of abdominal free fluid with the FAST examination has also been shown to decrease the time to surgical intervention and improve resource use.⁴ The FAST examination has been accepted as an adjunct to the secondary survey by Advanced Trauma Life Support since 1997.⁵ Point-of-care US is also used in the identification of abdominal free fluid in the acutely ill (nontrauma) patient. The presence of free fluid on US imaging, in conjunction with clinical acumen, can narrow the differential diagnosis in a critically ill patient and aid in decision making regarding the use of antibiotics, the need for surgical intervention, or decisions to transfer to another facility with tertiary care.^6–8

Performing and interpreting point-of-care US examinations require personnel with appropriate training and practice. The FAST examination is part of emergency medicine and surgical residency training programs,^9,10 and the overall use of US by emergency physicians who have recently graduated is increasing (FAST being the most common application).¹¹ Studies have shown that the FAST examination has high specificity for abdominal free fluid in both blunt and penetrating trauma.^12–28 However, physicians who are trained in point-of-care US may not be available at all hours or locations to acquire and interpret US images. Community emergency departments face many barriers to the use of point-of-care US, but most report that the primary reason for not using US is a lack of training.²⁹ There is also a need for accurate FAST interpretation in resource-limited international settings and prehospital settings, including emergency medical transport, mass-casualty and disaster response, and battlefield medicine.^30–36 In these settings, and without other diagnostic imaging available, the identification of free fluid by US would be a factor in determining the need for higher levels of care or medical evacuation.

Ultrasound image transfer for off-site interpretation has been shown to be feasible,^37,38 but it requires a dependable data connection and an on-site technician who is trained to acquire images correctly. A recent pilot study demonstrated that after a brief educational session, nonphysicians in the intensive care unit could obtain clinically useful US images with real-time tele-intensivist guidance.³⁹ However, in an emergency setting, an important benefit of a FAST examination is that it can be conducted in a matter of minutes. Telemedicine is not likely to replicate this rapid response. Robotic tele-manipulation of a US transducer by an off-site expert sonographer has been explored,⁴⁰ but robotic tele-manipulation inherently increases equipment cost and decreases portability.

Despite increases in point-of-care US clinical training, there is a need for a system to enable an operator with minimal training to perform a FAST examination. The development of such as system would require 2 core components: (1) instructions and feedback to guide an untrained user in the proper collection of a diagnostic US study; and (2) interpretation-assistance software to assist clinicians to accurately interpret the examination. The purpose of this pilot study was to evaluate the feasibility of automatically detecting abdominal free fluid on abdominal US images. Image-processing techniques for segmentation of US images exist in the scientific literature. Particular attention has been given to echocardiography, vascular disease (intravascular US), fetal US, and cancer screening, with a focus on prostate (transrectal US) and breast cancers.⁴¹ Automated detection of free fluid on abdominal US imaging presents a unique challenge because free fluid is amorphous. To our knowledge, this work is the first study to demonstrate automated detection of free fluid on abdominal US imaging.

Materials and Methods

Study Design and Setting

We conducted a retrospective pilot study using cross-sectional abdominal US videos from patients with normal and positive examination results for free-fluid from the Boston Medical Center Emergency Department Ultrasound Section image archive of emergency US examinations. These US examinations were performed for a variety of educational or clinical reasons within the Emergency Department at Boston Medical Center. This study was approved by the Boston University School of Medicine Institutional Review Board.

Study Population and Selection Criteria

FAST examinations for patients aged 18 years and older were retrieved from the database, regardless of sex and chief condition. Ten examinations with positive results and 10 examinations with negative results were randomly selected from the database after screening for image quality. For the purposes of this pilot study, perihepatic (ie, right upper quadrant) videos were used. Videos were deidentified, but medical record numbers corresponding to each US examination were used to look up basic patient data, including age, sex, and chief condition, from the electronic medical record.

Study Protocol

All frames from the selected US videos were subjected to the following analysis routine to identify hypoechoic regions of interest in the images using MATLAB (The MathWorks, Natick, MA). Hypoechoic regions of interest can represent abdominal free fluid but can also be blood vessels, the gallbladder, or other normal tissue.

Detection of Hypoechoic Regions of Interest

Each image was resized to 600 pixels in height, retaining the aspect ratio (typically 800 pixels in width). Borders of the field of view were manually identified, and all areas outside the view area were made black (eg, removing any labeling on the images and leaving only the US image). Next, a radial coordinate system was defined such that the medial and lateral borders of the US image were each of a constant radius, and the image was bisected into superior and inferior portions by the angle Θ = 0. The image was then sharpened to enhance edges (using the MATLAB function imsharpen).

Each image was then cropped as follows. A hyperechoic mask was defined (eg, a binary image of the same size and shape as the US frame with hyperechoic pixels assigned a value of true and all other pixels assigned a value of false) using the following routine: (1) increase contrast; (2) fill holes in the image (a hole is defined as a closed region of 0-valued pixels surrounded by non–0-valued pixels; (3) remove gray scale (eg, create a binary mask); (4) apply edge smoothing; (5) morphologically close the image; (6) apply the hole fill again; and (7) remove small shapes as defined by being less than 0.2% of the total image area. Next, a region of interest for the US image was defined by removing the following: (1) portions of the image medial to the most medial part of the hyperechoic mask; (2) shadowed areas (eg, near 0-valued pixels) along the superior and inferior borders; and (3) portions of the image lateral to the peritoneum. The purpose of this step was to create a moreblack-and-white (eg, contrasted) image and take out unnecessary objects to make further image-processing steps easier.

Shadows (eg, rib and gallstone shadows) were identified in 2 steps. First, large hypoechoic shapes were identified by adjusting the contrast, selecting pixels that were lighter than a threshold value, removing small shapes (as defined by being <2% of total image area), and eroding and then dilating the remaining shapes to break narrow bridges between shapes. Next, the length of the shape (eg, radial direction) and the width of the shape (eg, angular direction) in the previously defined radial coordinate system were evaluated. Shapes with a sufficiently high length-to-width ratio were identified as shadows. A shadow mask (eg, binary image) was defined spanning from 2 SDs below the mean of the angle values contained in the shape to 2 SDs above the mean and spanning the full radial distance from the lateral to the medial borders of the US image. The purpose of this step was to identify shadows so that they could be eliminated from further analyses.

Hypoechoic regions of interest were then identified by excluding shadows, adjusting the contrast, selecting pixels that were darker than a threshold value to form shapes, removing small shapes (as defined by being <0.1% of the total image area), and eroding and then dilating the remaining shapes to break narrow bridges between shapes. Next, hypoechoic regions of interest were rejected if: (1) they projected laterally from the medial border of the image or superiorly/inferiorly from the inferior/superior border of the image; (2) they shared more than 40% of their border with a shadow; (3) they were not sufficiently hypoechoic; or (4) the region just outside the border of the shape was not sufficiently more hyperechoic (eg, brighter) than the region immediately inside the border of the shape. The purpose of this step was to eliminate certain hypo - echoic regions (those that could be defined as not being free fluid based on certain descriptive characteristics) to improve the performance of the support vector machine classifier (see “Data Analysis”).

Finally, a common place for free fluid to collect is in the potential space between two organs or other pieces of tissue that touch each other. A characteristic of these, sometimes small but clinically important, fluid collections is that they are often slender, elongated, and bordered by hyperechoic areas. To identify these potential free-fluid collections the following steps were taken: (1) the US image was converted to a binary image by marking any pixels with a brightness of less than 20 (on a 0–255 scale, where 0 indicates pure black); (2) the binary image was opened linearly to extenuate elongated shapes; (3) 1-pixel bridges between shapes were removed; (4) very small shapes (<0.01% of the total image area) and large shapes (>0.5% of the total image area) were removed; (5) remaining shapes were reduced to image skeletons (ie, 1-pixel width representations of the shapes), separated at branch points, and dilated back to the original form; and (6) remaining shapes were rejected if the region just outside the border of the shape was not sufficiently more hyperechoic (ie, brighter) then the region immediately inside the border of the shape. The purpose of this step was to identify slender free-fluid collections in the potential space between two organs that would not otherwise be captured.

Manual Shape Classification

The hypoechoic regions of interest identified by the afore-mentioned procedure were then overlaid on top of the US videos. Physician investigators reviewed all positive examinations and categorized each hypoechoic region of interestas free fluid, not free fluid, or indistinguishable. Shapes labeled as indistinguishable accounted for 7.4% of all shapesand were excluded from further analysis. All hypoechoic regions of interest from the negative FAST examinations were categorized as not free fluid. Videos were down-sampled to 10 Hz before manual analysis to reduce the time burden.

Feature Selection

For all detected hypoechoic regions of interest in both the positive and negative examinations, the following features were computed:

Features related to geometric properties of the shape:

1. Linearity—the percentage of variance in the row/column positions of all pixels in a shape that was explained by the first principal component of the data (eg, using principal component analysis);

2. Curvilinearity—the standard deviation of the error in predicting a row position based on knowledge of a column position using a best-fit cubic polynomial;

3. Radius angle covariance—covariance of radius and angle values for each pixel in a shape;

4. Roundness—the standard deviation of the distances from each point on the border of a shape to the centroid of the shape (in row/column coordinates);

5. Position—the position of the centroid of the shape in the US field of view (in row/column coordinates); and

6. Area—the number of pixels comprising the shape.

Features related to grayscale color properties of the shape:

1. Echogenicity—the average of the grayscale color values of each pixel in the shape;

2. Echo Variability—the standard deviation of the grayscale color values of each pixel in the shape;

3. Medial neighborhood echogenicity—the average of the grayscale color values of each pixel in an area just medial to the medial border of the shape;

4. Medial neighborhood variability—the standard deviation of the grayscale color values of each pixel in an area just medial to the medial border of the shape;

5. Lateral neighborhood echogenicity—the average of the grayscale color values of each pixel in an area just lateral to the lateral border of the shape; and

6. Lateral neighborhood variability—the standard deviation of the grayscale color values of each pixel in an area just lateral of the lateral border of the shape.

Other features:

1. Edge sharpness—the average echogenicity of pixels just outside the border of the shape divided by the average echogenicity of pixels just inside the border of the shape; and

2. Number of hyperechoic pixels lateral to the shape but medial to the peritoneum.

Data Analysis

Features were tested for normality by the Kolmogorov-Smirnov test (α = .05). Features were grouped according to manual classification of the hypoechoic regions of interest (eg, positive or negative for free fluid). Finding that all features were non-normally distributed, Wilcoxon rank sum tests were used to compare median values for each feature between groups. Next, these features were normalized and used as inputs to a support vector machine classifier with a radial basis function using L1 soft-margin minimization by quadratic programming (fitcsvm.m; MATLAB). A 10-fold cross-validation by frames was performed. In other words, all frames from all videos were randomly distributed into 10 equally sized groups without repetition. For each group, a support vector machine classifier was trained on the remaining 90% of the data and tested on the set-aside 10% of data. Ten-fold cross-validation is a technique to reduce overtraining: that is, creating a model that works well for the data in question but is not well adapted to classify future samples. The support vector machine categorized each hypoechoic region of interest as free fluid (positive) or not free fluid (negative). Any frame with at least 1 shape that was categorized as free fluid was determined to be positive; all other frames were considered negative. Finally, a US video with at least 1 sequence of 2 or more positive frames was defined as positive, and all other videos were considered negative. All data analysis was performed in MATLAB.

Outcome Measures

The primary outcome measures were the sensitivity and specificity of the support vector machine classifier on a shape-by-shape, frame-by-frame, and patient-by-patient basis. Ninety-five percent confidence intervals for the primary outcome measures were computed in MedCalc Online version 15.10 (MedCalc Software, Ostend, Belgium).

Results

The characteristics of the study population, including age, sex, and chief condition, are shown in Table 1. In total, 4201 hypoechoic regions of interest were identified in 1264 frames from 20 right upper quadrant US videos (10 positive and 10 negative for abdominal free fluid). Of these, 2227 regions of interest from 669 frames were from positive cases, and 1974 regions of interest from 595 frames were from negative cases. The stages of the image segmentation process for 4 exemplary cases are shown in Figure 1.

Table 1.

Patient Demographics and Chief Conditions

Sex	Age, y	Chief Condition
Positive FAST exam
Male	52	Found down
Male	56	Fall from bicycle, left-side pain
Male	82	Abdominal pain
Male	50	Abdominal pain, nausea, vomiting
Male	50	Abdominal pain, alcohol abuse
Male	41	Stab wound
Unknown	16	Unknown
Female	41	Upper abdominal and rib pain
Unknown	Unknown	Unknown
Male	50	Abdominal pain
Negative FAST exam
Male	68	Lethargy
Female	30	Rectal bleeding, abdominal pain
Male	92	Abdominal pain
Male	44	Shortness of breath
Female	22	Abdominal pain
Male	27	Abdominal pain
Male	42	Motor vehicle collision
Male	37	Chest pain
Male	37	Shortness of breath
Male	64	Chest pain

Figure 1.

Right upper quadrant US images for 4 exemplary cases. Case 1: stab wound, splenic laceration, positive for free fluid. Case 2: found down, perforated gastric ulcer, septic shock, metabolic acidosis, positive for free fluid. Case 3: unknown injury mechanism, positive for free fluid. Case 4: abdominal pain, negative for free fluid. First row shows the original US images. Second row shows the images after sharpening. Borders of the field of view (shown in gray) were detected on the basis of identification of hyperechoic regions (highlighted with a whitening mask). Third row shows the detected shadows (outlined in gray). Fourth row shows the initially detected hypoechoic regions of interest before application of shape rejection rules. Fifth row shows the final hypoechoic regions of interest. Regions classified as positive by manual review are outlined in green and regions classified as negative by manual review are outlined in red. ROIs indicates regions of interest.

Fourteen features were calculated for each region of interest. All 14 features were non-normally distributed (Kolmogorov-Smirnov test, P < .05). Wilcoxon rank sum tests revealed significant differences (P< .05) between groups (eg, positive and negative for free fluid) in the median values for all features except curvilinearity, area, and the number of hyperechoic pixels lateral to the shape but medial to the peritoneum (Figure 2). Classification results by shape, frame, and patient (i.e., video) are shown in Tables 2–4, respectively.

Figure 2.

Probability density functions for each feature from regions of interest that were classified by manual review as positive for free fluid (solid green) or negative for free fluid (dashed red). P values for Kolmogorov-Smirnov tests are shown. Px indicates pixels.

Table 2.

Classification Results by Shape for the Support Vector Machine Classifier

	Actual
Predicted	Positive	Negative
Positive	434	9
Negative	223	1874

Sensitivity (95% confidence interval), 66.1% (66.3%–69.6%); specificity (95% confidence interval), 99.5% (99.1%–99.8%).

Table 4.

Classification Results by Frame for the Support Vector Machine Classifier

	Actual
Predicted	Positive	Negative
Positive	10	1
Negative	0	9

A video with at least 1 sequence of 2 or more positive frames was defined as positive. Sensitivity (95% confidence interval), 100% (69.2%–100%); specificity (95% confidence interval), 90.0% (55.5%–99.8%).

Discussion

This pilot study demonstrates the feasibility of developing a computer program that would automate the detection of free fluid in the FAST examination. Our algorithm was able to detect free fluid with sensitivity of 100% and specificity of 90%. Prior studies evaluating the performance of the FAST examination with human interpretation for free-fluid detection have shown sensitivity values of 63% to 100%and specificity values of 94% to 100% for blunt trauma.^12,14–17 The use of FAST in penetrating trauma remains more controversial; whereas specificity is on par with blunt trauma, sensitivity has been reported from 28% to 100%.^3,18–24 Ultrasound can also be used in the identification of abdominal free fluid in the acutely ill (nontrauma) patient,⁸ but studies of sensitivity and specificity are limited by the large variety of etiologies and the newness of using FAST in nontrauma patients.

Much work remains to be done before benefits to patient care can be realized. Our approach needs to be expanded to include all quadrants of the FAST examination. In this pilot study, we only considered perihepatic (ie, right upper quadrant) views. The algorithm will need to be tested in a larger sample and in a larger spectrum of disease (range of fluid volumes) to confirm the diagnostic performance of this test, ideally compared to a true-positive reference standard (computed tomography or laparotomy) as well as a US-interpreted standard. These future studies should include a separate assessment of test performance for different etiologies (eg, blunt trauma, penetrating trauma, and various nontrauma etiologies).

Automated free-fluid detection using US could have a broad and substantial impact on patient care across a wide range of medical specialties and settings and could be used for trauma and nontrauma (medical) applications. The advantages of US (including low cost and portability) make it an ideal triage tool for situations in which medical evacuation and response priorities must be determined (eg, mass casualties, battlefield, and other extreme environments).³¹ Similarly, such technology could reduce unnecessary patient transfers from rural emergency departments to tertiary care settings (a leading cause of these transfers is lack of radiologic services such as US42). Ultrasound can also be used in the identification of abdominal free fluid in the acutely ill (nontrauma) patient. In this context, automated abdominal free-fluid detection could be widely used: for example, by inpatient or outpatient internists looking for ascites. Finally, much like the automated electrocardiogram read, and even in settings with experienced sonographers on hand, automated abdominal free-fluid detection could assist providers in abdominal US interpretations. Taken together, the initial results presented in this study and the broad potential impacts on care delivery warrant further exploration of automated abdominal free-fluid detection using US.

There were several limitations of this study. First, the low sample size (n = 20) resulted in large confidence intervals of sensitivity and specificity. Second, 10-fold cross-validation of the support vector machine was performed by randomly distributing all frames from all videos into 10 equally sized groups without repetition. Future studies with larger sample sizes should use more rigorous cross-validation procedures that randomize data by patient instead of by frame. Given the limited number of patients and the large variety of clinical presentations, a “leave 1 patient out” cross-validation was not feasible in this study. Third, the image-processing algorithm used in this study ran in MATLAB, made use of existing MATLAB image-processing functions, was not optimized for speed, and processed image data that had been manually offloaded from the computer used to collect US images. For the proposed algorithm to be used in a timely manner in the setting of trauma, it must be embedded into US acquisition software and optimized for speed. This process is left for future work. Finally, the development of a US-based system that can be used by an operator with minimal training to rapidly detect abdominal free fluid would require, in addition to free-fluid detection algorithms, systems to guide an untrained user in the proper collection of a diagnostic US study. The feasibility of such a system should be tested in future studies. Physicians with US training can guide minimally trained nonphysicians to obtain high-quality, clinically useful US images through 2-way video,³⁹ However, it remains to be seen whether automated feedback and instructions can be used to guide a minimally trained operator without the need for an on-call expert and image transfer to an off-site facility.

In conclusion, this pilot study suggests that computerized detection of free fluid on abdominal US images may be sensitive and specific enough to aid clinicians in their interpretation of a FAST examination. This process could affect the practice of multiple medical specialties. Further technical development is required before this technology can be implemented, and further research is required to assess the impact on patient care.

Table 3.

Classification Results by Frame for the Support Vector Machine Classifier

	Actual
Predicted	Positive	Negative
Positive	286	6
Negative	96	435

Any frame with at least 1 shape that was categorized as free fluid was determined to be positive. Sensitivity (95% confidence interval), 74.9%(70.2%–79.1%); specificity (95% confidence interval), 98.6% (97.1%–99.5%).

Abbreviations

FAST

focused assessment with sonography for trauma

US

ultrasound