Computer-aided Assessment of Catheters and Tubes on Radiographs: How Good Is Artificial Intelligence for Assessment?

Abstract

Catheters are the second most common abnormal finding on radiographs. The position of catheters must be assessed on all radiographs because serious complications can arise if catheters are malpositioned. However, due to the large number of radiographs obtained each day, there can be substantial delays between the time a radiograph is obtained and when it is interpreted by a radiologist. Computer-aided approaches hold the potential to assist in prioritizing radiographs with potentially malpositioned catheters for interpretation and automatically insert text indicating the placement of catheters in radiology reports, thereby improving radiologists’ efficiency. After 50 years of research in computer-aided diagnosis, there is still a paucity of study in this area. With the development of deep learning approaches, the problem of catheter assessment is far more solvable. This review provides an overview of current algorithms and identifies key challenges in building a reliable computer-aided diagnosis system for assessment of catheters on radiographs. This review may serve to further the development of machine learning approaches for this important use case.

Supplemental material is available for this article.

© RSNA, 2020

Summary

Computer-aided assessment of catheters on radiographs will require integration of a variety of approaches to determine the presence, location of tip, the course, and the type of catheter; together, this information can be used to determine if the catheter is in a satisfactory position.

Essentials

• ■ Although several approaches have been used to assist with aspects of catheter assessment, including detection, classification, tracing, and assessment of tip location, there are currently no systematic approaches to solving the catheter placement evaluation problem in the published literature.

• ■ Available studies investigate only a subset of catheters and tubes while simply ignoring the others.

• ■ Previous studies evaluated their respective proposed method on a small, private dataset, and there are no large-scale reference standard datasets for proper evaluation of current automatic systems.

• ■ Algorithms have demonstrated the highest detection accuracy for endotracheal tubes, with lower detection accuracy achieved for nasogastric tubes and feeding tubes, likely due to the higher contrast and less variable course of endotracheal tubes.

• ■ We provide a new dataset with 100 neonatal chest and abdomen radiographs with catheters with corresponding pixelwise accurate annotation maps to help support this field of research.

Introduction

A variety of intravascular catheters, as well as gastrointestinal and airway tubes (all henceforth referred to as catheters for simplicity), are used in clinical practice for life-supporting purposes, especially in critically ill patients in intensive care units and emergency departments (1–6). For example, endotracheal tubes (ETTs) are used to assist in lung ventilation and may prevent aspiration; umbilical venous catheters may be used for administration of fluids or medications in neonates (7). For catheters to be used safely, proper placement is essential as serious complications can arise when a catheter is malpositioned. Radiography is routinely used to assess catheter positioning after catheter insertion because of the wide availability and cost-effectiveness of this imaging modality. However, because of the large number of radiographs obtained each day, there can be a substantial delay in time between when a radiograph is taken and when it is interpreted by a radiologist. Table 1 presents commonly used catheters, a brief description of each, and an estimate of the frequency with which they might require repositioning following a confirmatory radiograph.

Table 1:

Selected Catheters and Tubes Commonly Encountered on Chest Radiographs

A computer-aided detection (CAD) system that can automatically detect and localize catheter placement offers several potential benefits to radiologists and radiology departments. These benefits include appropriate prioritization of cases with potentially malpositioned catheters in worklists for interpretation, thereby shortening the turnaround time for critical cases, and automatic insertion of text, indicating the placement of catheters in radiology reports and thereby improving radiologists’ efficiency. Another possible application of computer-assisted approaches for catheter detection may be the processing of radiographs to remove all catheters from an image in preparation for analysis by a CAD system for detecting pathologic findings. This may be helpful as current CAD systems for detection of pathologic findings are often trained on images without any catheters present, and such structures may potentially be a source of confusion for deep learning algorithms (18).

Although the first CAD systems for evaluation of radiographs were proposed in the 1960s by Lodwick et al (19), it was not until 2007 that the first specialized system for catheter placement evaluation was published (20,21). In the last 12 years, there have been some efforts toward evaluating catheter placement on radiographs, but a general approach that is suitable for all types of catheters has yet to be presented. Early efforts in computer-aided catheter placement evaluation were limited by the feature representations of the catheter appearance and oversimplified assumptions of catheter geometric shape and position. Many of the related works use low-level image processing operations, such as template matching–based (22) region growing and Hough transform–based line fitting (23), which are often found to be fragile. On the other hand, radiographic features can exhibit many different forms of variation, as demonstrated in Figure 1, as well as extraneous structures such as electrocardiography lines or random coils outside of the body, which can further complicate evaluation. In addition, catheters can have low contrast-to-background structures, and window levels must often be adjusted to better visualize catheters. Recently, there has been a paradigm shift from traditional rule-based approaches to machine learning–based or, more specifically, deep learning approaches in medical imaging, resulting in drastic improvement of both diagnostic accuracy and interpretation time.

Figure 1a:

Three examples of radiographs show the variety of lines and the complexity of catheter shape and appearance: (a) an adult chest radiograph with a right-sided chest tube, (b) a neonatal chest and abdomen radiograph with an endotracheal tube (ETT), nasogastric tube (NGT), umbilical arterial catheter (UAC), and umbilical venous catheter (UVC), and (c) a neonatal chest and abdomen radiograph with a coiled umbilical arterial catheter, an umbilical venous catheter, and an endotracheal tube.

Figure 1b:

Three examples of radiographs show the variety of lines and the complexity of catheter shape and appearance: (a) an adult chest radiograph with a right-sided chest tube, (b) a neonatal chest and abdomen radiograph with an endotracheal tube (ETT), nasogastric tube (NGT), umbilical arterial catheter (UAC), and umbilical venous catheter (UVC), and (c) a neonatal chest and abdomen radiograph with a coiled umbilical arterial catheter, an umbilical venous catheter, and an endotracheal tube.

In this article, we review the progress toward computer-assisted catheter placement evaluation algorithms and identify key issues and future opportunities in designing such systems. This review article will serve to further the development of machine learning approaches for this important use case, highlight particular challenges and opportunities, and provide radiologists and data scientists with an overview of how current CAD systems for catheter detection work and what they may achieve in the future.

Review

To be clinically useful, the systems that evaluate catheter placement should be able to answer the five questions shown in Figure 2. We begin with the most straightforward question of whether a catheter is present on the radiograph and then gradually extract more information, including the location of the catheter tip and catheter course (sequence of points as a representation of the catheter). With this information, we can start to identify the type of catheter based on both its appearance and geometric features (the catheter’s course). Finally, it is critical to assess whether the catheter is in a satisfactory position. Note that there are currently no general algorithms that can answer all the questions mentioned above for all types of catheters in one single system. We provide a brief overview of the traditional pipeline of the catheter placement evaluation systems as shown in Figure 3 and mark on the top left corner of each relevant step the relevant question it tries to answer.

Figure 2:

Research questions to be answered for a system to evaluate catheter placement.

Figure 3:

Conventional catheter placement evaluation pipeline (drawn in black). The question numbers indicated in top left corner of each box correspond to the questions posed in Figure 2. Steps connected using magenta lines offer an alternative pipeline to answer all five research questions. Note that steps marked with a dashed line currently have no published research. CNN = convolutional neural network, ROI = region of interest, SRCNN = scale recurrent CNN, SVM = support vector machine.

In the following sections of this article, we categorize the related literature into five sections, each corresponding to the five questions that need to be answered for an automatic catheter placement evaluation system. At the end of each section, we describe the metrics used in the evaluation of these algorithms and identify potential future research directions and challenges.

Q1: Is a Catheter Present?

Determining the presence of a catheter should be the initial step. It is reasonable to assume that there is no prior knowledge about which type of catheters might be present. In machine learning, this is a typical supervised binary classification problem. A curated, labeled dataset is needed for the algorithm to learn, with one class representing radiographs with catheters and the other class representing radiographs with no catheters. Lakhani trained a deep convolutional neural network (CNN) from end to end on 180 images to determine the presence or absence of ETT on chest radiographs (24). This study achieved an area under the curve (AUC) of 0.99, which is sufficiently accurate given the reasonably easy nature of this task, although there might be a risk of overfitting on this relatively small dataset.

Evaluation metrics.—The AUC—the integral measure of the receiver operating characteristic (ROC) curve—is a commonly used metric for binary classification problems in which positive and negative classes are balanced (25). When positive and negative classes are imbalanced, a precision-recall curve, which plots the positive predictive value against the true-positive rate, more accurately illustrates an algorithm’s performance. A related metric, the F1 score, is calculated as the harmonic mean of precision and recall.

Q2: Where Is the Tip of the Catheter?

Locating the catheter tip is critical for evaluation of catheter placement. Catheter tips should be placed in specific anatomic regions to ensure they function correctly and to minimize the risk of complications. For example, the tip of an ETT should be placed in the trachea well above the carina to minimize the risk of selective bronchial intubation, preventing the collapse of the contralateral lung. Generally, two methods of localizing the tip of the catheter are described in the literature. The first is by tracing from an initial seed point, either manually selected or automatically detected, until a criterion is fulfilled (26,27) (eg, a sudden intensity drop in the searching direction). The second is by using heuristics. For example, Lee et al proposed a supervised deep learning–based method to locate the tip of a peripherally inserted central catheter (PICC) (28). They segmented out the lung regions and PICC and defined the tip as the lowest endpoint of the PICC inside the lung region. In addition to PICCs, these approaches may also be applied for locating the tips of other types of catheters, such as tunneled central venous catheters and catheters connected to ports, as the distal aspects of these catheters have a similar appearance. A variety of conditions may result in the failure of these approaches, including coiling of the catheter and abnormal positioning of the catheter relative to anatomic landmarks, such as when the tip is superior to the lungs. A third possible approach that holds the most promise for catheter tip detection is direct regression as commonly used in general object detection using a deep learning approach. For catheter tip detection, regression targets would be the tip location rather than the four corners of the bounding box as used for general object detection (29).

Evaluation metrics.—The most common metrics used to evaluate the accuracy of tip localization are the mean absolute distance and the root mean square distance between the detected and the ground-truth tip location as marked by a radiologist (28). An alternative measure is the tip detection ratio as proposed by Keller et al (20). This is defined as the ratio of the detected catheter tips that are within a predefined distance (such as 2.5 mm as used by Keller et al) of the ground-truth location.

Q3: What Is the Course of the Catheter?

Simply ensuring the correct placement of the catheter tip is not always sufficient for assessment, as a catheter may loop on itself or take other aberrant paths. An example is shown in Figure 1c. In this case, knowing the complete course of the catheter is helpful for assessment. To accomplish this task, a starting seed must be selected. This has been previously accomplished by either manual selection (20) or automatic selection with template matching in the neck region of a radiograph (30).

Figure 1c:

Three examples of radiographs show the variety of lines and the complexity of catheter shape and appearance: (a) an adult chest radiograph with a right-sided chest tube, (b) a neonatal chest and abdomen radiograph with an endotracheal tube (ETT), nasogastric tube (NGT), umbilical arterial catheter (UAC), and umbilical venous catheter (UVC), and (c) a neonatal chest and abdomen radiograph with a coiled umbilical arterial catheter, an umbilical venous catheter, and an endotracheal tube.

A sequence of points on the catheter can be generated by a template matching–based region growing method. This method involves computing the cross-correlation of the image data around the seed with a predefined catheter intensity profile and finding the next point along the direction that gives the best fit, iterating this process until a stopping criterion is met (20,30,31). Instead of using template matching in the intensity domain, Sheng et al used the Hough transform on the local window of the edge image to determine which direction to trace (32). The downside of this operation is that the computation complexity would prevent it from being used in a real-time environment. Kao et al and Chen et al used an even simpler method, wherein the next point was determined on a row basis (26,27). The x location of the next point x_next is the one with the highest intensity values among [x_current − 1, x_current, x_current + 1].

All of the literature that involves catheter tracing assumes that the target catheter does not form loops. This is a valid assumption for ETTs but not for other types of catheters. Therefore, Yi et al, Mercan and Celebi, and Lee et al formulated this as a segmentation problem and modeled the catheter as a collection of points with no particular order (28,33,34). All three works adopted CNNs trained on their especially curated dataset. Mercan and Celebi used a patch-based CNN to segment chest tube and used nonuniform rational basis spline curve fitting to connect discontinuous catheter segments (34). Lee et al used a fully CNN to segment PICCs and used a probabilistic Hough transform to postprocess the potential discontinuities (28). Yi et al used a scale recurrent CNN to segment ETTs, nasogastric tubes (NGTs), umbilical arterial catheters, and umbilical venous catheters (33). It treated all catheters of interest as a single class and iteratively refined the segmentation results. Although promising results were achieved, they have only answered part of the question: Where are the catheters? The other part of the question is to differentiate each instance of catheter from the detected results. Deep learning approaches then may also be used to determine catheter coiling, which could be achieved by supervised training of a deep learning system on instance segmentation maps.

Evaluation metrics.—All of the studies that have computed a catheter course to determine the tip location did not explicitly evaluate the accuracy of the catheter course with the exception of the study by Keller et al (20). In that study, the authors computed the tracking accuracy and precision based on the tracked catheter midline and the ground-truth catheter midline annotated by radiologists.

The typical evaluation metrics for binary segmentation are the pixel-based ROC and precision-recall curve. Following catheter instance segmentation, a line fitting could then be performed to determine the catheter course. To evaluate the accuracy of the catheter course, we can borrow metrics used in segmentation that measure the boundary accuracy, such as mean absolute distance and Hausdorff distance.

Important aspects to consider when reporting evaluation metrics for catheter detection are the diameter of the catheter, the pixel resolution of the image, and the contrast-to-noise ratio of the images (related to the type of radiographic equipment, the radiographic parameters employed as well as the x-ray absorption of the materials from which the catheters are made), as each can affect the performance of an automated catheter detection system.

Q4: Which Type of Catheter Is It?

Ramakrishna et al conducted preliminary studies to differentiate NGTs and ETTs. The authors assume NGTs and ETTs have different intensity profiles and can therefore be identified via simple template matching (31). This approach does not work with umbilical arterial catheters and umbilical venous catheters on pediatric radiographs because both catheters have the same appearance and only differ in their course relative to anatomic landmarks. Geometric features need to be taken into consideration along with appearance features for this differentiation. Another approach for differentiating NGTs and ETTs was proposed by Sheng et al (32). They assume ETTs always lie between the two lungs. Kao et al and Chen et al extracted two simple features from the traced catheter and used a support vector machine classifier to classify a catheter as an ETT or NGT (26,27).

Evaluation metrics.—Because classification of catheters on radiographs is essentially a multiclass classification problem, a confusion matrix—a type of contingency table—can be computed, and per-class classification accuracy can be determined. In addition, per-class AUC can be determined, assuming that determining the type of each catheter is a binary classification problem (one-vs-all strategy).

Q5: Is the Catheter Appropriately Positioned?

Determining whether a catheter is appropriately positioned relative to anatomic landmarks is critical for the safe use of catheters. Singh et al used Inception V3, ResNet50, and DenseNet in a TensorFlow framework to classify enteric feeding tube placements on chest and abdominal radiographs as critical (ie, bronchial insertion) or noncritical (ie, with the feeding tube in the esophagus, stomach, or duodenum; appropriately coursing beyond the field of view; or absent from the radiograph). Inception V3 (pretrained with color images from ImageNet and trained using radiographs labeled as critical or noncritical enteric tube placements) had the highest AUC overall, with an AUC of 0.87 (95% confidence interval: 0.80, 0.94) (35). To our knowledge, no reports regarding assessing the position of multiple types of catheters on a single radiograph exist.

Evaluation metrics.—Similar to Q1: Is a catheter present?, determining whether a catheter is appropriately positioned is generally a binary classification problem, and the AUC can appropriately describe performance when positive and negative classes are balanced.

Literature regarding computer-aided evaluation of catheter placement as described above is summarized in Table 2. As further algorithms are developed and studies describing their performance are published, a meta-analysis of performance measures may be warranted.

Table 2:

Summary of Publications on Computer-aided Evaluation of Catheter Placement

Current Limitations and Areas for Further Development

Although several approaches have been used to assist with aspects of catheter assessment, including catheter detection, classification, tracing, and assessment of tip location, there are no clear systematic approaches to solving the placement evaluation problem. Previous studies have investigated only a subset of the types of catheters that are commonly found on radiographs, ignoring the others. The proposed algorithms may demonstrate acceptable performance on the dataset the respective authors curated, but it is difficult to conclude that the algorithm will show sufficient performance at other institutions without further application to other large-scale datasets with a wide variety of cases and catheter profiles. Among all the catheter types, algorithms have demonstrated by far the highest detection accuracy for ETTs, likely because they consistently lie in the neck and upper chest and have high contrast to the background. NGTs and feeding tubes, on the other hand, are much more challenging to detect because they are obscured by the cardiac silhouette and abdominal structures and their more variable position. Whether performance can be maintained when ETTs are combined with other catheters remains to be determined.

Although catheters are frequently present on radiographs (38), there has been remarkably little overall progress in the use of machine learning approaches for their evaluation. One of the impediments to further development is the lack of large annotated datasets to establish a reliable reference standard, not only for catheters but also for reference landmark organs such as the lungs, heart, and vertebral bodies. However, there is already some collective effort to make a large number of radiographs available for the research community to use. For example, the ChestX-ray14 dataset containing 112 120 radiographs with 14 image-level class labels referring to chest abnormalities was released in 2017 by the National Institutes of Health (39). These datasets contain radiographs with catheters so they can be annotated in a way that is suitable for catheter placement evaluation research. Such datasets may be annotated with “strong” or “weak” labels (40). Strong labels generally refer to pixel-level annotation, including pixel-level annotation of the catheter and, in some cases, landmark organs used as a reference to assess catheter location. Weak labels refer to labels for the entire image, such as a label that an ETT is present or that an ETT is appropriately positioned or malpositioned. Annotating images with strong labels requires more human effort; however, it is anticipated that considerably less training data are required when using strong labels.

Annotating line structures using pixel-level annotations (strong labels) takes considerably more effort than other anatomic or pathologic regions. Besides pixel maps, there are other ways of annotation that require less time and effort, such as drawing polygonal bounding boxes. There are some tools designed for this purpose, such as Polygon-RNN++ (41). However, because catheters are generally not localized and can span the whole radiographic image, bounding boxes are not as useful as they are in other general computer vision problems. Research productivity could be significantly boosted with the development of dedicated easy-to-use tools for line structure annotation.

A related area for further development is real-time catheter segmentation and catheter tip detection during image-guided procedures, such as interventional radiology procedures or voiding cystourethrography for example. Real-time catheter segmentation or tip detection may aid radiologists in tracking catheters to ensure they are advanced to the correct locations without inadvertently creating injury during catheter insertion. Real-time catheter detection may also aid in co-registration between two- and three-dimensional imaging modalities for three-dimensional real-time virtual navigation systems, automated respiratory motion compensation, and automatic collimation to the region of the catheter tip (42). Real-time catheter segmentation during fluoroscopic procedures poses several challenges, including the low signal-to-noise of fluoroscopic images, relatively high frame rate (eg, 7–30 frames per second), and the fact that as the catheter is manipulated it may change considerably in location and shape between two consecutive frames. However, early work has shown promising results. Ambrosini et al adapted a U-Net model to segment a catheter and guidewire using the current image and the three previous images as inputs. They achieved tip detection with a median tip distance error of 0.9 mm. The median centerline distance error was 0.2 mm, and 85% of the frames were within 1 mm of centerline distance error (43). Wu et al used a cascaded detection-segmentation model consisting of detection and segmentation CNNs to segment catheter tips on fluoroscopic images obtained during percutaneous coronary intervention procedures. Their approach achieved tip precision of 0.532 pixels, an F1 score of 0.939, false tracking rate of 0.800%, and missing tracking rate of 9.900% (44). However, the running speed of 4–5 frames per second using two Nvidia TITAN Xp 12G graphics processing units (Santa Clara, Calif) is a current limitation to applying this approach more broadly in clinical practice. Using a conventional machine learning approach, Milletari et al sought to automatically detect electrophysiology catheters in fluoroscopic sequences using Laplacian of Gaussian and difference of Gaussian–based filters, a Top-Hat filter to discard candidates that do not fulfill spatial and geometric constraint characteristics of an electrode, clustering, and scoring criteria using a greedy algorithm. Their approach achieved a tip detection rate of 99.3% and a mean distance of 0.5 mm from manually labeled ground-truth centroids. However, the approach relies on knowing a priori the number of catheters on the image (45).

Another challenge facing the interventional radiology community is perceived time lag (latency) between the production of images and the true position of the catheter as it is manipulated. Although increasing the frame rate allows for a more accurate indication of real-time positioning, it comes at the cost of increased radiation exposure (46). Deep learning approaches for image denoising and catheter detection may allow users to use a higher frame rate and lower dose rate, potentially reducing radiation dose overall while maintaining high image quality. In addition to catheter detection at conventional fluoroscopy, there is also opportunity to develop catheter detection systems for MRI (particularly in the context of MRI-guided interventional procedures) and multiprojection x-ray systems, such as biplane imaging and tomosynthesis. Although limited work has been conducted regarding catheter detection at CT, research may benefit from borrowing approaches used for detecting tubelike structures, such as the airways, at CT imaging (47).

In summary, computer-aided assessment of catheters on radiographs will require integration of a variety of approaches to determine whether a catheter is present, where the tip of the catheter is, what the course of the catheter is, which type of catheter it is, and, ultimately, whether the catheter is in a satisfactory position. Although current research is still in the early phases, we anticipate seeing additional work related to catheter detection in the coming years because of the ubiquity of catheters on radiographs and the potential clinical benefits of an automatic catheter placement evaluation system. Integration of an automatic catheter placement evaluation system into the radiology workflow may promote patient safety and assist radiologists in generating accurate reports in a shorter amount of time. Given that there is no public dataset for chest and abdomen radiographs in neonates—many of whom have a wide variety of catheters and tubes—and to support efforts in building a generalized solution for catheter detection and attract the attention of a wider variety of researchers, we are releasing a locally collected neonatal chest and abdomen radiograph dataset that contains 100 images and catheter annotation maps with this article (https://github.com/xinario/pediatricxray100). Further details regarding the dataset are provided in Appendix E1 (supplement).