Lung Nodule Detection from Feature Engineering to Deep Learning in Thoracic CT Images: a Comprehensive Review

Abstract

This paper presents a systematic review of the literature focused on the lung nodule detection in chest computed tomography (CT) images. Manual detection of lung nodules by the radiologist is a sequential and time-consuming process. The detection is subjective and depends on the radiologist’s experiences. Owing to the variation in shapes and appearances of a lung nodule, it is very difficult to identify the proper location of the nodule from a huge number of slices generated by the CT scanner. Small nodules (< 10 mm in diameter) may be missed by this manual detection process. Therefore, computer-aided diagnosis (CAD) system acts as a “second opinion” for the radiologists, by making final decision quickly with higher accuracy and greater confidence. The goal of this survey work is to present the current state of the artworks and their progress towards lung nodule detection to the researchers and readers in this domain. This review paper has covered the published works from 2009 to April 2018. Different nodule detection approaches are described elaborately in this work. Recently, it is observed that deep learning (DL)-based approaches are applied extensively for nodule detection and characterization. Therefore, emphasis has been given to convolutional neural network (CNN)-based DL approaches by describing different CNN-based networks.

Introduction

Cancer is a disease that has several manifestations and it is primarily associated with abnormal cell groups. These cancer cells continue to divide and grow to produce tumors. Among all types of cancer, lung cancer is the most life-threatening disease all over the world. According to the World Health Organization [1], lung cancer is the leading cause of death worldwide. In 2008, 1.37 million deaths caused by lung cancer occurred throughout the world [2]. Available data shows that the lung cancer is the largest among new cancer diagnoses worldwide (1,350,000 new cases and 12.4% of total new cancer cases) and it is the largest cause of death from cancer globally (1,180,000 deaths and 17.6% of total cancer deaths) [3].

It is the most common cancer in men worldwide (1.1 million cases, 16.5% of the total). Among females, it was the fourth (516,000 cases, 8.5% of all cancers) most commonly diagnosed cancer and the second (427,000 deaths, 12.8% of the total) leading cause of cancer death [4]. Last 5 years (2014-2018) estimated new lung cancer cases and total estimated lung cancer deaths found in the USA [5] have been portrayed in Fig. 1. A report by National Cancer Centre Singapore [6] has shown that 1.2 million people were diagnosed with cancer and 7700 people died of cancer on average in the year 2015 and 158,080 cases were expected by the end of 2016. In Brazil, an average of 28,220 cases found in 2016 with 17,330 males and 10,890 females [7].

Fig. 1.

Last 5 years (2014–2018) lung cancer statistics in USA. a Nos. of estimated new cases. b Total nos. of estimated deaths. Source: American Cancer Society (ACS) [5]

Lung cancer can be divided into two broad categories, viz., small cell lung cancer (SCLC) and non-small cell lung cancer (NSCLC). SCLC accounts for 15% of lung cancer cases and highly malignant. Non-small cell lung cancer (NSCLC) accounts for the remaining 85% of cases [3].

A pulmonary nodule refers to the lung tissue abnormality mostly found in lung cancer patients. In 1984, a glossary of terms published by Fleischner Society [8] for thoracic radiology. The society has defined lung nodule as “any pulmonary or pleural lesion represented in a radiograph by a sharply defined, discrete, nearly circular opacity 2-30 mm in diameter.” Twelve years later, the society defined lung nodule as “round opacity, at least moderately well marginated and no greater than 3 cm in maximum diameter.”

The solitary pulmonary nodule (SPN) is defined as a radiographic opacity with a diameter of up to 30 mm and at least two-thirds of its margins surrounded by lung parenchyma. SPNs are of the four types: (a) well-circumscribed: the nodule is located centrally in the lung without significant connections to vasculature, (b) vascularized: the nodule is located centrally in the lung, but has significant vascularization (connections to neighboring vessels), (c) juxtapleural: a significant proportion of the nodule periphery is connected to the pleural surface, and (d) pleural tail: The nodule is near the pleural surface, connected by a thin structure called “pleural tail” [9]. Figure 2 shows four types of SPNs. On the other hand, if the diameter is greater than 30 mm, it is called lung mass. Lung masses are generally cancerous.

Fig. 2.

Typical SPNs for different types. a Well-circumscribed nodule. b Juxtavascular nodule. c Nodule with a pleural tail. d Juxtapleural nodule (Dhara et al. [17])

It is observed from different research works that early detection of lung nodules can improve the 5-year survival rate. Among different types of cancer, the 5-year survival rate of lung cancer is the lowest with an estimated value of 18% of the diagnosed cases for years 2004-2010. The prognosis of lung cancer depends on the extent of disease at the time of diagnosis. Early-stage detection of lung cancer can improve the survival rate. The 5-year survival rate is 54% for a small and localized lesion [10]. According to Cancer Research UK [11], the survival rate is 87% for stage I, and 19% for stage IV lung cancer patients.

Many pitfalls of recent medical imaging techniques have been investigated by the work of Godoy et al. [12] for nodule detection and characterization. Computed tomography is the most recent dominated imaging tool used to capture the lung images. The National Lung Screening Trial (NLST) [13] reported that a reduction of 20% in mortality can be achieved with low-dose computed tomography (LDCT) image than chest X-ray. Therefore, computer-aided detection and computer-aided diagnosis has become the most prominent and valuable tool for lung nodule detection and characterization.

A complete guideline for small pulmonary nodule management including its growth information is provided by the work of MacMahon et al. [14]. Recent guidelines and management of pulmonary nodules are described in [15]. A deep and extensive study of patients with previous cancer history was presented by the work of Rena et al. [16] for SPN characterization.

The morphological appearances such as border, shape, and location characteristics of a lung nodule in CT image are described in [18-20]. The information about ground glass opacity (GGO) nodule is available in [21]. Different types of GGOs are shown in Fig. 3.

Fig. 3.

Examples of a sub-solid and b pure ground-glass nodules (Lederlin et al. [15])

Reviews of the pulmonary detection techniques are not new but as the computational techniques change from time to time, a thorough review is necessary to understand the technology shipment towards automated nodule detection systems. Different review works with objectives to find out the technological changes exist in the literature [17, 22-27].

However, owing to the developments of new techniques, it has been noticed that a recently deep learning-based approach has introduced for nodule detection. Therefore, a detailed survey is required to cope up with the technological changes and present consistent information to the thirsted readers, the aim of this survey work.

This survey work primarily conducted by focusing on the “nodule detection” process from CT images. Therefore, different works based on nodule detection have been searched and collected from IEEE, Elsevier, SPIE, and Springer. The following sentences were used as search terms for handcraft- (HC) or feature engineering-based works: (1) lung nodule detection, (2) nodule detection, (3) lung nodule detection using filters, (4) lung nodule segmentation, and (5) nodule candidate detection. On the other hand, sentences such as (1) lung nodule detection using deep learning, (2) lung nodule detection using deep features, (3) lung nodule detection using convolutional neural network, and (4) lung nodule detection using CNN have been used for deep learning-based works.

It is observed that, apart from nodule detection, CNN-based nodule characterization/classification tasks are also available in the literature. CNN acts as an automatic feature extractor, owing to this reason the nodule characterization becomes much easier. The following search terms have been used to collect different works based on the nodule characterization, viz., (1) lung nodule characterization, (2) lung nodule classification, (3) lung nodule characterization using CNN, and (4) lung nodule classification using CNN.

This review paper is organized as follows. The “Database Description” section presents a detail description of the available databases. The “Traditional/Handcraft/Feature Engineering-Based Approaches Towards Lung Nodule Detection” section describes different nodule detection frameworks based on handcraft/feature engineering approach. The “Deep Learning-Based Approach Towards Lung Nodule Detection” section is devoted to the deep learning-based approach towards nodule detection and characterization. Finally, the reported survey work has been concluded in the “Conclusion” section.

Database Description

In this step, CT images are collected for evaluation of the CAD systems. In most of the nodule detection works, the images used have dimension 512 × 512 pixels in the form of a 2-D slice. A number of slices are varying in different scans. All images use Digital Imaging and Communications in Medicine (DICOM) file format. The high-resolution CT (HRCT) images can be acquired from the few publicly available databases, e.g., LIDC, ANODE, and from different hospitals (private database). Below, a brief introduction has been given for the two publicly available databases.

(a) LIDC-IDRI

The Lung Image Database Consortium-Image Database Resource Initiative [28] is the world’s largest publicly available database that acts as a leading source of nodule reference database. This image repository was initiated and developed by five institutions in USA, viz., Weill Cornell Medical College, University of California (Los Angeles), University of Chicago, University of Iowa, and University of Michigan [29]. Readers are referred to the work of Armato III et al. [29] and Tan et al. [30] for the historical perspective of the LIDC-IDRI database. The database contains 1018 CT scans of 1010 patients. The thickness of each slice varies from 1.25 to 2.5 mm range with the pixel size ranges between 0.48 and 0.72 mm [31]. Most of the work used the nodule size between 3 to 30 mm (11-113 pixels). A detailed description of the LIDC-IDRI can be found in [29, 32]. The works of Gupta et al. [31], Hancock and Magnan [33], Han et al. [34], Hancock and Magnan [35], Messay et al. [36], and Paul et al. [37] have presented the marking techniques, evaluation methods, and guidelines to use of this database.

(b) ANODE09

The Automatic Nodule Detection 2009 (ANODE09) is another publicly available database consists of 55 CT scans, annotated by two radiologists. Out of 55 scans, 5 were provided as training cases, and the rest 50 cases were provided as the testing case. The details of this database can be found by the work of Ginneken et al. [38].

Apart from LIDC and ANODE09, three more databases, viz., TIME [39], ELCAP [40], and LISS [41], are available in the public domain for lung nodule research. These databases are small in size w.r.to LIDC.

Traditional/Handcraft/Feature Engineering-Based Approaches Towards Lung Nodule Detection

This section describes the workflow of a computer-aided detection (CADe) system by explaining different works based on “Feature Engineering” for nodule detection. This traditional framework consists of four main steps: (1) pre-processing, (2) lung segmentation, (3) nodule detection, and finally (4) classification (Fig. 4). Each step has been described precisely by considering the related works that have been found so far.

Fig. 4.

A simple CADe framework

Pre-processing

The aim of this step is to reduce noise and enhancement of nodule-like structures by applying different filters to the input lung CT image. Different filters can be found in literature, among them median filter, dot enhancement filter, log filter, and filters based on histogram equalization are widely used in this stage. Median filter has been used in [42-46]. The work of Sun et al. [47] and Choi and Choi [48] used dot enhancement filter. Log filter was used by the work of Nagata et al. [49], Jirapatnakul et al. [50], and Kubo et al. [51]. Wiener filter was applied in [52]. CLAHE and Wiener filter have been used in [53]. Multi-scale filter was examined by the work of Pei et al. [54] and Takemura et al. [55]. Selective enhancement filter has been used by Miyajima et al. [56] and Dhara and Mukhopadhyay [57]. Dhara et al. [58] applied a multi-scale selective enhancement filter. Gabor filter was used by the work of Hadavi et al. [59]. Filho et al. [45] and Namin et al. [60] have used Gaussian filter in the pre-processing stage. Apart from that, Iris filter [61], shell filter [62], sequential filter [63], fuzzy filter [64], and LBP filter [65] have been used as a pre-processing step.

Lung Segmentation

The second step of the nodule detection system is the lung segmentation. This stage aims to reduce the search space by extracting only the lung regions. This step mainly includes the following sub-steps thorax extraction and parenchyma structure extraction/lung extraction.

Thorax Extraction

This stage processes and removes all artifacts external to the patient’s body, e.g., bed sheets, the air that involves him, and the surface on which he lies [66].

Lung Extraction

The overall aim of this module is the identification of the left and right lung from a CT slice. The histogram of the lung CT image with two sharp peaks from left to right has been shown in Figs. 5 and 6, respectively.

Fig. 5.

Gray-level distribution of lung CT image (Cascio et al. [67])

Fig. 6.

Threshold may be obtained by choosing the valley of the histogram (Sousa et al. [66])

A typical thoracic CT slice contains voxel intensities in Hounsfield unit (HU) scale. It is observed from the above histograms that low-density lung tissues are visible in the intensity range of − 910 to − 500 HU, whereas high-density structures such as chest wall, blood, bone, fat, and muscles contain intensity value above − 500 HU. In most of the works, researchers have incorporated threshold-based segmentation, as there is a sharp valley in the histogram (Fig. 6). Experimentally, a value of − 500 HU may be chosen as the ideal threshold value. Otherwise, an iterative optimal threshold algorithm may be used to determine an optimal threshold. Therefore, by designing a proper mask, it is possible to extract the left and right lung region from CT image. A region-growing algorithm with a threshold as stopping criteria may be used for lung segmentation as mention in [66]. A 3-D region-growing algorithm has been used by several studies for lung segmentation. After segmentation, a 3-D labeling algorithm labels the image. Finally, two lung regions have been extracted as the biggest and next biggest connected regions from the labeled image and setting the rest of the segmented regions as background [68, 69]. Lung segmentation based on threshold can be found by the work of Leemput et al. [62], Cuifang et al. [64], Suárez-Cuenca et al. [61], Sousa et al. [66], Santos et al. [68], Zhang et al. [70], Fu et al. [71], Gopi and Selvakumar [72], Paing and Choomchuay [73, 74], Javaid et al. [75], Rendon-Gonzalez and Ponomaryov [76], Xu et al. [65], Saien et al. [77], Choi and Choi [78, 79], Pu et al. [80], and Messay et al. [81]. Segmentation based on iterative threshold [48, 82, 83], Otsu threshold [73, 84], adaptive threshold [61, 85], and 3-D adaptive fuzzy threshold [86] exist in the literature. Lung segmentation based on region and 3-D region growing can be found in [42, 67, 87-92]. Zhang et al. [70] have introduced a global active contour model for lung segmentation. Figure 7 shows an example output of the lung segmentation stage.

Fig. 7.

Output of “lung segmentation” stage. Segmented left and right lung appear in slices 40 and 21, respectively (Santos et al. 2014 [66])

Nodule Detection

This section describes the main module of a CADe-based system. The aim of this module is the detection of potential region of interests (ROIs), here lung nodule using image processing techniques. The term, “nodule detection” has been used in a broad sense and consists of the following two sub-stages, viz., (1) nodule candidate detection/tubular structure elimination and (2) feature extraction/false-positive reduction. The complete framework for the CADe system has been shown in Fig. 4. Lee et al. [25] have categorized the nodule detection scheme as (a) segmentation detection, (b) classification detection, (c) segmentation-template detection, and (d) segmentation-classification detection. It has been observed that in the traditional nodule detection approach, almost all the works are following the segmentation-classification-based approach. The aim of nodule detection module is to identify true nodule candidates from the false positives (FPs), such as blood vessels, shades, bronchioles, bifurcation points, and ribs by using different image processing and pattern recognition techniques. Next, use or invent different discriminating features so that nodules can be detected (Fig. 8) with higher classification accuracy. In this section, different journal works and few conference-based works have been described elaborately for nodule detection found so far from the time period 2009 to April 2018.

Fig. 8.

Output of “nodule detection” stage. a Original lung CT image, input to the CADe system. b Detected nodule

Nodule Candidate Detection/Tubular Structure Elimination

Image segmentation divides an image into different disjoint regions. This process changes the representation of an image into some meaningful image regions for better understanding and analysis in the subsequent stages. Therefore, this process reduces the search space within an image and at the same time finds the location of the desired objects for identification purpose. A good survey on the segmentation category and medical image segmentation can be found by the work of Pal and Pal [93], Haralick and Shapiro [94], Phamy et al. [95], and Sharma and Aggarwal [96]. The segmentation approaches can be divided into three generations. The first-generation approach includes different algorithms depending on the pixel intensity and different low-level image processing techniques. Segmentation involves edge and thresholding and region-based approaches are examples of this category. First-generation algorithms do not incorporate any prior information of the image. Second-generation algorithms introduce uncertainty models and different optimization techniques for image segmentation. Segmentation based on clustering, genetic algorithm (GA), neural network (NN), and partial differential equations are examples of second-generation segmentation. Third-generation algorithms incorporate high-level knowledge such as “a piece of prior” information; expert knowledge and knowledge about the object shape, orientation, continuity, elasticity, or smoothness are included in the segmentation process. Segmentation based on deformable model, active appearance model (AAM) and atlas-based approaches are examples of third-generation segmentation algorithm. Different segmentation algorithms have been introduced for accurate nodule segmentation or nodule candidate detection or both. Research works focused on nodule segmentation can be found in [36, 86, 97-101]. Our aim of this survey is to emphasize on the nodule detection process. In this subsection, different segmentation and detection frameworks have been discussed for nodule candidate detection.

The frameworks have been categorized based on different segmentation approaches. Segmentation based on thresholding [66, 78, 79, 81, 102-104], region-based [105], clustering-based [45, 75, 106-109], mixed or hybrid segmentation algorithm-based [110], partial differential equation-based [99, 111], model-based [67, 112] and atlas-based [113] methods were reported in the literature for nodule candidate detection. Apart from that, some other approaches, e.g., template matching based [88, 114], filtering based [48, 57, 62, 115-118], feature based [60, 119], concept based [42, 80, 120, 121], and works based on specialized algorithms [122-125] are available in the literature for complete nodule detection framework.

A. Thresholding-Based Methods

This approach is the simplest one among different image segmentation approaches. Image histogram is used to partition the image. Each peak of the histogram represents a particular region and the intensity value between two peaks called the “threshold” is selected to separate desired classes. Next, the segmentation process is done by grouping all pixel intensities w.r.to. threshold value. A good survey on thresholding-based segmentation can be found by the work of Sahoo et al. [126]. Sousa et al. [66] segmented internal structures of the lung by choosing appropriate threshold values. After that, tubular structures were eliminated by applying the 3-D skeletonization algorithm. In their work, researchers have observed that tubular structure, e.g., vessel shows a low variation of the average depth w.r.to medial axis. On the other hand, nodules show an abrupt increase of values from the nodule border. Therefore, by choosing appropriate threshold value with skeletonized segmentation algorithm nodules has been separated from vessels. Messay et al. [81] used intensity thresholding and morphological processing techniques for nodule candidate detection. Choi and Choi [78] have identified initial nodule candidates by using thresholding and shape information. Narayanan et al. [102] have segmented and detected candidate nodules simultaneously using intensity-based thresholding combined with morphological processing. Aresta et al. [103] developed a nodule detection framework using Otsu’s threshold method. Different solid, sub-solid, non-solid, and juxtapleural nodules have been detected by their proposed system. Solid nodules were searched slice-wise inside a fixed-width sliding window with stride 1. Next, the Otsu threshold method has been applied inside each window for candidate generation. Finally, all windows were combined using logical OR operation. The sub-solid and non-solid nodule candidates were detected using a Laplacian-of-Gaussian (LoG) filter combined with Otsu’s method. Mehre et al. [104] combined intensity threshold with structuring elements (SEs) for candidate nodule detection. Morphological opening operation was performed by using six different intensity values combined with radii of SEs, viz., (− 400, 2), (− 500, 2), (− 600, 2), (− 400, 3), (− 500, 3), (− 600, 3) for nodule candidate detection. Choi et al. [79] applied optimal multiple thresholding and rule-based pruning method for nodule candidate detection and segmentation.

It is observed that the detection sensitivity varies from 82.66% with an average 3 FPs/scan/case [81] to 94.1% at 5.54 FPs/scan [79] by using threshold-based approach. The juxtapleural nodule candidates were also detected with a sensitivity of 57.4% at 4 FPs/scan by using this approach [103].

The thresholding approach is entirely based on pixel intensity of an image. It highly depends on the histogram peaks of an image. Finding an appropriate threshold is a challenging task; as pixel intensity is more sensitive to noise and intensity inhomogeneities. Apart from these, spatial characteristics of an image are not taken into account by this approach and not good for images with wide plane valleys and unclear histogram peaks.

B. Region-Based Methods

This approach partitions an image into different regions by satisfying some homogeneity criteria. Gray-level pixel values are used as homogeneity criteria. Region-based segmentation includes different techniques, viz., region merging, region splitting, and split and merge. “Split and Merge” is a hybrid approach, taking the advantages of both methods. In this approach, pixels are grouped by satisfying some similarity criterion. The selection of the similarity criterion plays an important role in accurate segmentation. A survey on region-based segmentation is presented in Freixenet et al. [127]. Suárez-Cuenca et al. [105] developed a 3-D region-growing algorithm for accurate nodule segmentation. Initially, the nodule candidates were detected by a selective enhancement filter and thresholding approach. The researchers have detected 71.8% of nodules at 0.8 false positives per case, 75.5% at 1.6 FPs/case, and 80% at 3.4 FPs/case, respectively. The region-based algorithm gives poor results as compared with a threshold-based nodule detection approach.

The primary disadvantage of region-based approach is that it requires careful manual interaction for seed point initialization as well as proper seed planning for the regions that need further processing, although the split and merge technique does not require a seed point. Apart from that, different extracted regions may contain holes or even disconnected. These algorithms are also sensitive to noise.

C. Clustering-Based Methods

This method is based on the division of pixels into homogeneous clusters. This is an unsupervised approach to image segmentation method. It finds the natural grouping of pixels by executing certain criterion function. Different clustering algorithms, e.g., K-means or ISODATA algorithm, fuzzy c-means (FCM), and the expectation-maximization (EM) algorithm, have been applied extensively for medical image segmentation. Filho et al. [45] proposed a CADe system based on the quality threshold (QT) algorithm and the diversity index. The QT algorithm was used for nodule candidate detection. Javaid et al. [75] have used the K-means algorithm for nodule detection. Initially, the number of classes has been chosen as three. Next, the class with the highest mean intensity value was chosen as the nodule cluster. Juxtavascular nodules are attached to the blood vessels. Therefore, the separation of blood vessels from the Juxtavascular nodule is a crucial task. The researchers have categorized these attachments (connection) as 2-D and 3-D type. If vessels were connected to a nodule in a single slice, it was called “2-D connection.” On the other hand, a“3-D connection” was defined as the branches and nodules that appear to be separated objects on a single slice but were attached in any previous or some next slice. Researchers have used a shape-specific morphological opening operation to break the 2-D connection. A structuring element “line” with parameters length and angle has been used by their study. Removal of 3-D connected branches has been done by comparing the areas of 3-D connected regions on each slice. Finally, all nodule candidates were grouped into six classes by using the proposed equations, viz., “nodule inhomogeneities thickness” and “% of wall connectivity.” Santos et al. [68] developed a CADe system for automatic small nodule (2-10 mm) detection on CT images. After lung segmentation, the internal structures have been segmented using the Gaussian mixture model (GMM) and EM algorithm. Then a Hessian matrix was used for false-positive reduction. However, it has been observed that the presence of bronchi bifurcations could not be removed by using only a Hessian matrix. Nithila and Kumar [106] have presented a new technique that incorporates particle swarm optimization (PSO) algorithm with back-propagation neural network (BPNN). Nodules were initially detected by using FCM clustering. The data points have been clustered into three different intensity levels; low, medium, and high. Spherical objects have been identified using roundness rule, and tubular structures were removed using elongation rule. Farahani et al. [107] developed a type-II fuzzy algorithm for the quality improvement of raw CT images. Then, modified spatial kernelized fuzzy c-means (MSFCM) clustering algorithm has been used for internal structure segmentation. After that, morphological opening, closing, and filling operations have been used for nodule candidate detection. Netto et al. [108] developed a CADe system based on growing neural gas (GNG) clustering algorithm. Initially, candidate nodules were detected using GNG algorithm. Next, nodule candidates were further regrouped using 3-D region-growing algorithm for volume reduction. Hosseini et al. [109] proposed an automatic system that can learn and tune Gaussian interval type-2 membership functions (IT2MFs). Finally, the system has been applied for lung nodule detection.

It is observed that the detection sensitivity varies from 85.91% with 1.82FPs/exam [45] to 93.2% [107] by using the clustering-based method. An accuracy of 98% (for solid nodule), 99.5% (for part-solid nodule), and 97.2% (for non-solid nodule) respectively has been achieved by the work of Nithila and Kumar [106]. Farahani et al. [107] also have achieved an accuracy of 96.5% by using this approach. As compared with the previous threshold and region-based detection approaches, the detection accuracy of clustering approach is higher with lower false positives.

Most of the clustering algorithms require initial parameters. The EM algorithm has greater sensitivity to initialization than the K-means or FCM algorithm. This approach does not consider spatial modeling and can be sensitive to noise and intensity inhomogeneities. The lack of information and uncertainty in data can be dealt with the FCM algorithm by using a membership function. Fuzzy if-then rules could be utilized to do approximate inference. However, choosing or designing an appropriate membership function is not a trivial task and calculations involved in fuzzy approaches could be intensive.

D. Mixed or Hybrid Segmentation Algorithm-Based Methods

This approach consists of combining different segmentation approaches by using some criterion function. Antonelli et al. [110] designed a new CADe system that includes three sub-modules (1) image segmentation, whose aim is to extract pulmonary regions; (2) VOI (voxel of interest) detector, looking for VOIs inside the pulmonary regions; and (3) nodule detector, classify the VOIs into nodules and non-nodules. Three segmentation algorithms, viz., robust fuzzy c-means (RFCM), iterative threshold, and region growing, have been used in parallel to the initial segmented image for VOIs detection. A combination scheme (VOI detector combiner) has been used to merge the outputs of the algorithms to form the VOIs list. This VOI list acts as a nodule candidate locator in their proposed system. Finally, five classifiers, viz., linear classifier (ldc), quadratic classifier (qdc), logistic classifier (loglc), decision tree (treec), and a radial basis function neural network (rbnc), have been used for nodule detection. The proposed system has achieved a sensitivity of 92.5%. It is observed that the sensitivity value is high for mixed algorithm-based system but choosing the proper combination criteria is difficult as different segmentation algorithm belongs to a different generation and follows their philosophy.

E. Partial Differential Equation-Based Methods

This approach uses partial differential as its basic set of equations. Active contour model (ACM) and level-set method (LSM) belong to this category. Another name for ACM is snake. A snake is an energy minimizing, deformable spline. It is defined by two energy terms, viz., internal elastic energy and the external edge-based energy. Internal energy term controls the deformations applied to the snake, and the external energy term controls the fitting of the contour onto the image. The finally used energy function is the sum of the external energy and internal energy terms; it acts like a snake. Level-set methods are a conceptual framework and act as a tool for numerical analysis of surfaces and shapes. Keshani et al. [99] have presented a technique using ACM. In the first step, binary lung CT images were obtained by adaptive fuzzy thresholding. After that, false positives were reduced by applying two different sized windows, immediately followed by another two rotational windows. Next, two threshold values were used to improve the result and making an initial mask for the ACM model. A global region-based energy function that uses mean intensities has been used by their work for nodule candidate segmentation and detection. Gong et al. [111] have used a unique 3-D tensor-filtering algorithm combined with local features for nodule candidate detection. Next, a 3-D level-set method has been used to correct and refine the boundaries of nodule candidates. A sensitivity of 79.3% was achieved by their work. An overall detection rate of 89% with 7.3 FPs/scan has been achieved by the work of Keshani et al. [99]. The disadvantages of ACM include it is not parameter free, sensitive to local minima states, and accuracy highly depends on the convergence policy. On the other hand, LSM is a parameter-free approach. In LSM technique, constructing appropriate velocities for a level-set function is difficult.

F. Model-Based Methods

This approach assumed that the structure of the organs has a repetitive form of geometry and then it is modeled probabilistically for variation of shape and geometry. Active shape model (ASM), active appearance model (AAM), and deformable models are examples of model-based segmentation methods. A survey on deformable models can be found by the work of McInerney and Terzopoulos [128]. Cascio et al. [67] proposed a new algorithm based on the 3-D mass-spring model. The proposed system can detect small-sized (3 mm diameter) nodules. The stable 3-D mass-spring model (MSM) combined with spline curves has been used for nodule segmentation. Farag et al. [112] have used AAM with a template matching technique. Four templates were obtained using AAM for four categories of nodule detection. Cascio et al. [67] have achieved a detection rate of 97% with 6.1 FPs/CT with a sensitivity of 88% with 2.5 FPs/CT.

In model-based segmentation, manual interaction is required to place an initial model and choose appropriate parameters, the main disadvantage of this approach. Apart from that, this approach gives poor convergence to concave boundaries [96].

G. Atlas-Based Methods

This approach incorporates prior information of the object. Different expert knowledge or statistical information extracted from the available examples. The a prior knowledge is incorporated in both atlas construction and atlas-fitting procedure. In this approach, different features are computed using an atlas or lookup table. Alam et al. [113] have used an atlas-based segmentation method for lung nodule detection. In the first step atlas, pre-segmented image has been selected using nine feature descriptors and the K-nearest neighbors (K-NN) algorithm. Then in the second step, nodule candidates were selected using a patch-based image segmentation method. Finally, the Laplacian of Gaussian (LoG) filter has been applied for accurate nodule segmentation and detection. Researchers have reported a sensitivity of 100% by using the atlas-based approach.

The main advantage of this approach is that it can perform segmentation and classifications in one go [96]. The disadvantage of an atlas-based approach is the time necessary for atlas construction.

H. Template Matching-Based Methods

This is the “brute-force” algorithmic approach for object detection process. A template is a sub-image contains the region of interest (ROI); slides over the entire input image for the search of desired objects. Cross-correlation, normalized cross-correlation (NCC), sum of absolute difference (SAD), and sum of squared differences (SSD) are the metrics, commonly used for the matching process. Gonga et al. [88] developed a new algorithm based on dynamic self-adaptive template matching and Fisher linear discriminant analysis (FLDA) classifier. Initially, the lung volume was segmented by using Otsu and the 3-D region-growing algorithm. Next, a multi-scale Gaussian smoothing operation has been operated for noise reduction and gradient calculation. After that, the 3-D dot enhancement filter has been applied over the smooth image. Finally, a threshold was applied on the dot-enhanced image to get the nodule candidate regions. Then the spherical ROIs have been obtained by using a 3-D dot-filtering algorithm. Next, a 3-D dynamic self-adaptive template matching algorithm has been used with the middle slice of 3-D ROI as the gray distribution of the 3-D template. Researchers have used the maxima of NCC and SSD matching methods for nodule candidate detection. Then a 3-D region-growing algorithm has been applied for accurate nodule segmentation. Farag et al. [112] proposed a template matching technique for nodule detection. Four templates were designed by their work. Each template represents the mean shape and texture of one of the four types of nodules. These models have been generated using an AAM method that allowed for obtaining a more realistic texture and shape descriptor of the nodules. NCC has been used as the similarity measurement. Assefa et al. [114] have combined a template matching algorithm with a multi-resolution feature analysis technique for nodule detection and false-positive reduction.

Initially, nodule candidates were detected by using a template matching approach. Finally, authors have used NCC as a similarity measure for nodule candidate detection. A sensitivity of 90.24% with 4.54 and FPs/scan in LIDC and 84.1% with 5.59 FPs/scan in ANODE09 database has been achieved by the work of Gonga et al. [88]. Farag et al. [112] also have achieved a sensitivity of 85%. It is observed that sensitivity in the range of 84-91% has been achieved by using template matching approach.

The main demerit of template matching approach is the proper choice of metric for matching with the target image. The algorithms, e.g., NNC, SSD, SAD, are mostly time consuming, taking more time to compute the correlation. NCC involves the time-consuming division, square root operations for correlation computation and hence is more complicated as compared with both SSD and SAD algorithms.

I. Filtering-Based Methods

In this approach, different filters have been designed and processed for accurate nodule segmentation and candidate detection. Most of the filters have been designed based on the eigenvalues of the Hessian matrix. Another name of the Hessian matrix is second-moment matrix; derived from the gradient of a function. It is a square matrix and can be decomposed using eigenvalue decomposition; gives a linear combination of three eigenvalues with three eigenvectors. These three components provide explicit structural information about the surfaces, curvedness, and pointedness of an object that can be used for nodule candidate detection. Choi and Choi [48] detected nodule candidates by using a proposed multi-scale dot enhancement filter. Hessian matrix has been used to calculate the dot values of the filter. The proposed system has shown a maximum accuracy of 97.4% with a high sensitivity value of 97.5%. Jaffar et al. [115] also have used multi-scale filter based on the eigenvalues of Hessian matrices for nodule candidate detection. Retico et al. [118] applied a multi-scale dot enhancement filter combined with peak detector algorithm for internal nodule candidate detection. Yokota et al. [129] developed a system that can detect GGO nodules from lung CT images. A 3-D line filter using the Hessian matrix has been used for vessel removal. Next, GGO candidate nodules were detected using density and gradient information. Finally, GGO candidate nodules were segmented using a suitable threshold. Ye et al. [86] have used volumetric shape index map along with the Hessian matrix for nodule detection. The proposed system has achieved a sensitivity of 91.3% for solid nodule and 83.3% for GGO nodule with an average detection rate of 90.2% with 8.2 FPs/scan. Novo et al. [116] also have used Hessian matrix for nodule candidate detection. A new technique named as central adaptive medialness has been used along with the Hessian matrix by their work. This technique has been applied over the bright object regions where the sum of the eigenvalues was less than zero. The main advantage of this method is that it does not require any Gaussian smoothing operation. The method is more sensitive to small nodules. Chen et al. [117] proposed two new features for false-positive reduction. A blob-like structure enhancement (BSE) filter has been introduced for nodule detection. The “blobness” feature was computed by utilizing Hessian matrix and has applied across multi-scale fashion in the lung CT image. It is observed that Hessian-based BSE filter has low sensitivity in the boundary region; results in under-segmentation. Therefore, a segmentation technique named as “fine segmentation” has been introduced for the inclusion of the boundary voxels. A LoG filter has been used in a bounded region for candidate segmentation. Dhara and Mukhopadhyay [57] have investigated a hybrid pre-processing technique using geometry based diffusion (GBD) filter and selective enhancement filter for nodule detection. In the first step, GBD filter has been used for FP reduction. Next, a selective enhancement filter has been used for false-negative reduction. Leemput et al. [62] have proposed a new filter for nodule detection, named as shell filter. After lung segmentation, gray-scale dilation operation was performed with a spherical shell structuring element. Next, voxel wise minimum was computed between the filtered image and original lung segmented image. Finally, candidate regions were detected by taking the difference between the resulted image and the original image.

It is observed that sensitivity in the range of 88.65-97.5% has been achieved by filter-based approach. However, designing and choosing a good filter that can suppress vessel-like structure and enhance blob-like structure is a difficult task.

J. Feature-Based Methods

This approach is composed of computing different features for candidate nodule identification. Shape index (SI) and curvedness (CV) are the two most important features that have been used by feature-based methods. This strategy was originally proposed by Murphy et al. [119], where the researchers first down-sample the image and then apply a single Gaussian filter with σ = 1. Next, the SI and CV features were computed using the principal curvatures k₁ and k₂ for nodule candidate detection. The researchers have achieved a sensitivity of 80% with an average of 4.2 FPs/scan. Namin et al. [60] also applied shape index feature and achieved a sensitivity of 88% with 10.3FPs/subject. Ye et al. [86] used volumetric shape index feature for nodule candidate detection. Next, dot-like candidate regions were enhanced using a multi-scale dot enhancement filter. The proposed CAD system was able to detect both solid and GGO nodules from the CT image. The researchers have achieved a sensitivity of 91.3% for solid nodule and 83.3% for GGO nodule.

It has been observed that maximum sensitivity was achieved by the work of Ye et al. [86]. Therefore, these features can be combined with some other features, e.g., Hessian matrix based and texture based, for accurate nodule detection.

K. Concept-Based Methods

In this subsection, different concept-based approaches have been discussed for nodule candidate detection. Froz et al. [120] have used a concept based on artificial Life. A hierarchical vector quantization (VQ) scheme was adopted by the work of Han et al. [121]. Concepts based on the “Gestalt Psychology” principle and “Break-and-Repair” can be found in [42, 80]. Froz et al. [120] presented a new algorithm based on the concept of artificial crawlers (AC) and rose diagram (RD) for nodule detection. AC was defined as a model that uses two-dimensional images and lives as an individual in the pixels of gray-level images. The researchers have used candidate curve and template curve by employing the AC algorithm. In their work, nodule and non-nodules were used to generate the candidate curve, and only true nodules were used to generate the template curve. Next, a feature vector was formed by taking the differences between the template and candidate curve and employing area under the curve. Five distance functions, viz., Euclidean, simple matching, Jaccard, Chebyshev, and Manhattan have been used for each template. Next, different features, e.g., mean direction, circular variance, circular standard variation, mean resulting length, kurtosis, and skewness have been computed by employing the RD. Finally, nodules were detected by combining different features computed from the AC model and RD model. Han et al. [121] have introduced a new algorithm for the CADe system based on a hierarchical VQ scheme. Vector quantization was originally used in the field of data compression and signal processing and then become popular in the fields of speech recognition, face detection, image compression, classification, and segmentation, as mentioned by the researchers. In the reported work, the lung fields were segmented using a “high level” VQ methodology. The final lung mask was obtained using connected component analysis, followed by a flood-fill operation for filling up the holes and other disconnected regions. In the second step, a “low level” VQ algorithm has been used for nodule candidate detection and segmentation. Qiu et al. [42] proposed a CADe system based on the Gestalt psychology principle; states that people first focus on things as a whole and then understand the details view of things with the help of global understanding. Researchers have used comprehensive images, obtained by a linear combination of maximum intensity projection (MIP) image and the original lung CT image. Next, the axial, coronal, and sagittal comprehensive CT images have been combined for candidate detection. Pu et al. [80] have introduced a new methodology named as “Break-and-Repair.” The strategy consists of three steps: (1) “modeling,” (2) “break,” and (3) “repair.” In the first stage, the gross ROIs have been extracted and represented as a surface model. After that, non-anatomical structure regions were identified and removed in the second stage. Next, in the third stage, an implicit function has been used for “repairing” (estimating) the removed suspicious regions. In the “break” stage, the principal curvatures and the principal directions were calculated from the triangle mesh of the cube. Candidate nodules have been identified using Koenderink’s shape classification based on a “visibility” testing procedure that followed by curvature analysis. Finally, seed points satisfying the “visible” condition and have classified as convex-cylinder types by the curvature analysis were removed as false positives. Qiu et al. [42] have achieved an accuracy of 91.29%. An overall sensitivity of 82.7% with 4 FPs/scan and 89.2% at 4.14 FPs/scan has been reported in [121] for juxtapleural nodule detection. Pu et al. [80] achieved an overlapping ratio of 69.91 ± 9.43% by using the “Break-and-Repair” strategy.

L. Special Algorithm-Based Methods

This approach is based on designing some specific algorithms for nodule candidate detection. Taşci et al. [123] presented a new CADe system using the α-hull method for juxtapleural nodule detection. The candidate regions are defined as the region between lines obtained by applying α-hull and boundaries obtained by the lung segmentation. The accuracy of the proposed system was 95.88%. Retico et al. [125] reported a CADe system using 3-D directional-gradient concentration (DGC) algorithm followed by a morphological opening operation for nodule detection. Finally, a candidate list was prepared by using a peak detector algorithm. The FP rejection ability of the system was up to 99% as reported by the researchers. Yuan et al. [122] proposed a semi-supervised learning algorithm for automatic detection of GGO nodules from different CT slices. Radiologist only provides a mark in one slice, and the location of the GGO nodule in different adjacent slices will be detected automatically; reduce the time of GGO candidate nodule selection time significantly. Recognition rate of the proposed system was in the range from 91 to 100% as reported by the researchers. Wang et al. [124] reported support vector machine (SVM) classifier based on the 3-D matrix pattern technique that can prevent the loss of structural and local information. The researchers have achieved an overall sensitivity of 98.2% with 9.1 FPs per section. It has been observed that the average sensitivity of the systems using these specific algorithms is quite high and produces good results with higher detection accuracy. Table 1 summarizes different nodule detection systems using feature engineering with reported best performance in reverse chronological order.

Table 1.

Different nodule detection system using feature engineering approach in lung CT images

Sl. No.	First author with publication year	Nodule detection process	Performance
1.	Gong et al. (2018) [111]	3-D tensor-filtering algorithm with local feature analysis followed by 3-D level-set segmentation	Sensitivity = 79.3% at an average of 4FPs/scan in LUNA16 database and sensitivity = 84.62% at an average of 2.8 FPs/scan in ANODE09 database
2.	Farahani et al. (2018) [107]	Kernelized fuzzy c-means (MSFCM) clustering algorithm followed by morphological operation	Accuracy = 96.5%, sensitivity = 93.2%, specificity = 98.1%
3.	Zhang et al. (2017) [70]	3-D skeletonization based feature, voxel removal rate (VRR)	Avg. accuracy = 93.6%, sensitivity = 89.3% with 2.1 false-positive rate (FPR) per subject
4.	Alam et al. (2017) [113]	Atlas-based segmentation method	Sensitivity = 100%
5.	Narayanan et al. (2017) [102]	Intensity-based thresholding with morphological processing	Specificity of 3 false positives per case/patient on average and sensitivity in CT image is 87.86%
6.	Jaffar et al. (2017) [115]	Multi-scale filter based on the eigenvalues of Hessian matrices	Accuracy = 98.7% and sensitivity = 97.5%
7.	Aresta et al. (2017) [103]	(1) Solid nodules are detected slice-wise inside a fixed-width sliding window with stride 1 followed by the Otsu threshold and logical OR operation. (2) Non-solid nodules are detected using LoG filter	Sensitivity is 57.4% with 4 FPs/scan
8.	Yuan et al. (2017) [122]	Semi-supervised based learning algorithm for automatic detection of GGO nodules	Recognition rate is in the range from 91 to 100%
9.	Qiu et al. (2016) [42]	Gestalt psychology principle	Accuracy = 91.29%
10.	Froz et al. (2016) [120]	Artificial crawlers and rose diagram–based method	Mean accuracy (mACC) = 94.30%, mean sensitivity (mSEN) = 91.86%, and mean specificity (mSPC) = 94.78% and mean area under the receiver operating characteristic (mROC) = 0.922
11.	Mehre et al. (2016) [104]	Combinations of thresholds and structuring elements	Sensitivity = 92.91% with 3 FP/scan
12.	Nithila and Kumar (2016) [106]	Fuzzy c-means clustering with three clusters, viz., low, medium, and high	Accuracy is 98% for solid nodules, 99.5% for part-solid nodule, and 97.2% for non-solid nodules
13.	Javaid et al. (2016) [75]	K-means algorithm	Accuracy = 96.22%, sensitivity = 91.65% with 3.19 FPs per case, and sensitivity = 83.33% for small size nodule
14.	Gonga et al. (2016) [88]	3-D dot filtering combined with dynamic self-adaptive template matching algorithm	Sensitivity = 90.24% with 4.54 FPs/scan in LIDC dataset, sensitivity = 84.1% with 5.59FPs/scan in ANODE09 dataset
15.	Taşci and Uğur (2015) [123]	α-hull method	Accuracy = 95.88% and area under curve (AUC) = 0.9679
16.	Leemput et al. (2015) [62]	Shell filter	Overall system score was 0.336
17.	Novo et al. (2015) [116]	Central adaptive medialness technique with Hessian matrix	Sensitivity = 88.65%,
18.	Yokota et al. (2015) [129]	Density and gradient information	Accuracy = 93.0%
19.	Han et al. (2014) [121]	“Low level” vector quantization (VQ) algorithm	Overall sensitivity = 82.7%,specificity = 4 FPs/scan, and sensitivity = 89.2% at 4.14 FPs/scan for juxtapleural nodules
20.	Ciompi et al. (2014) [133]	Bag-of-Frequencies (BoF) feature descriptor	Area under the ROC curve for spiculation (Az) = 0.907 (for axial),0.903 (for sagittal), and 0.911(for coronal) images
21.	Santos et al. (2014) [68]	Hessian matrix with Tsallis’s and Shannon’s Q entropy features	Accuracy = 88.4%, sensitivity = 90.6%, specificity = 85% and false positives per exam is 1.17
22.	Filho et al. (2013) [45]	Quality threshold (QT) algorithm	Accuracy = 97.55%, sensitivity = 85.91%, specificity = 97.70%, with a false positive rate of 1.82 per exam and 0.008 per slice and area under the free-response operating characteristic is of 0.8062
23.	Choi and Choi (2013) [48]	Multi-scale dot enhancement filter with angular histogram of surface normals (AHSN) feature	Accuracy = 97.4%, overall sensitivity = 97.5% with 6.76 FPs/scan, and specificity = 97.7%
24.	Keshani et al. (2013) [99]	ACM segmentation followed by 3-D averaging feature	Overall detection rate is 89% with 7.3 false positives per scan
25.	Wang et al. (2013) [124]	3-D matrix pattern technique	Overall sensitivity = 98.2% with 9.1 FPs per section
26.	Assefa et al. (2013) [114]	Template matching algorithm with multi-resolution feature analysis technique	The detection rate is 81.212%
27.	Cascio et al. (2012) [67]	Stable 3-D mass-spring model (MSM) is combined with spline curves	The detection rate of the system is 97% with 6.1 FPs/CT. A reduction to 2.5 FPs/CT is achieved at 88% sensitivity
28.	Netto et al. (2012) [108]	Growing neural gas (GNG) clustering algorithm	The methodology ensures that nodules of reasonable size be found with sensitivity = 86%, specificity = 91%, and a mean accuracy of 91%
29.	Choi et al. (2012) [79]	Optimal multiple thresholding and rule-based pruning	94.1% sensitivity at 5.45 false positives per scan
30.	Soltaninejad et al. (2012) [46]	2-D stochastic features and 3-D anatomical features	90% detection rate with 5.63FPs/scan
31.	Dhara and Mukhopadhyay (2012) [57]	Geometry based diffusion (GBD) and selective enhancement filter	Volumetric overlap (VO) mean = 0.98 (solid nodule) and 0.93 for GGO nodule. Hausdorff distance (HD) mean = 9.4 (solid nodule) and 7.8 for GGO nodule
32.	Chen et al. (2012) [117]	Blob-like structure enhancement (BSE) filter followed by an LoG filter	Average TPR = 93.6% with 12.3FPs/case
33.	Farag et al. (2011) [112]	Template matching technique with AAM method	Sensitivity = 85% and specificity = 99% using SIFT feature descriptor
34.	Hosseini et al. (2011) [109]	Gaussian interval type-2 membership functions (IT2MFs)	Average ROC accuracy is 95% with a 99% confidence interval (CI) of [92–99]%
35.	Suárez-Cuenca et al. (2011) [105]	3-D region-growing algorithm	71.8% of nodules detected at 0.8 false positives per case, 75.5% at 1.6 FPs/case, and 80% at 3.4 FPs/case, respectively
36.	Antonelli et al. (2011) [110]	Combined the output of three segmentation algorithms (1) robust fuzzy c-means (RFCM), (2) iterative threshold, and (3) region growing	Sensitivity = 92.5% and specificity = 83.5%
37.	Pu et al. (2010) [80]	“Break-and-Repair” technique	RMS error (mm) 1.08 ± 0.45 and overlapping (%) 69.91 ± 9.43
38.	Messay et al. (2010) [81]	Combined intensity thresholding and morphological processing	Sensitivity = 82.66%, with an average of 3 FPs per CT scan/case
39.	Namin et al. (2010) [60]	Shape index (SI) feature	Sensitivity = 88%, with 10.3 FPs per subject
40.	Sousa et al. (2009) [66]	3-D skeletonization algorithm	Sensitivity = 84.84%, specificity = 96.15%, and accuracy = 95.21%
41.	Retico et al. (2009) [125]	3-D directional-gradient concentration (DGC) algorithm, followed by a morphological opening operation	Sensitivity = 72% and the FP rejection ability of the system is up to 99%
42.	Murphy et al. (2009) [119]	Shape index and curvedness features	Sensitivity = 80%, with an average of 4.2 FPs per scan
43.	Ye et al. (2009) [86]	Shape index map and dot map	Average detection rate of 90.2%, with approximately 8.2 FP/scan
44.	Retico et al. (2009) [118]	Dot enhancement filter and the “normals” to the pleura surface technique	Sensitivity in the 80–85% range is achieved with an average number of 6–9 FP per scan in their dataset and 67% sensitivity is achieved at 8 FP per scan in ANODE09 database
45.	Choi and Choi (2009) [78]	Thresholding and shape information	85.11% detection rate with 1.13 FPs per scan

Feature Extraction and False-Positive Reduction

In this stage, different intensity-based, morphological /shape-based, and texture-based features are extracted as feature vector from the detected nodule candidate regions and fed to a classifier for true nodule detection. Generally, nodules have a circular shape in 2-D CT images and spherical shapes in 3-D CT image, i.e., nodules have a compact structure. It follows Gaussian distribution and the region is more or less convex. Therefore, different geometrical and intensity-based features have been extracted for true nodule detection. A list of features is available in [75, 81, 121, 130-132] used for nodule detection. Sousa et al. [66] applied different morphological, textural, and intensity-based features for nodule detection. Messay et al. [81] identified nodules by extracting 245 features from candidate nodules for false-positive reduction. After feature extraction, features were selected using sequential forward selection (SFS) technique. Finally, Fisher linear discriminant (FLD) classifier and a quadratic classifier have been used for nodule detection. Choi and Choi [78] have used principal component analysis (PCA) for false-positive reduction. Narayanan et al. [102] computed a list of 503 features for nodule detection. Mehre et al. [104] have reduced false positives by using different structural and intensity-based statistical features. Different structural features, e.g., XY-width, Z-extent, elongation, cube compactness, pondered radial distance, and Boyce-Clark radial shape index have been used by their work. Choi and Choi [79] have extracted a set of 2-D and 3-D features for nodule detection purposes. Filho et al. [45] have used two diversity indexes, viz., Simpson’s index and Shannon’s index as features for nodule detection. Javaid et al. [75] grouped nodules into six classes by using their proposed equations, viz., “nodule thickness” and “% of wall connectivity.” Next, different features were extracted specific to tiny, small, and big group nodules. Tiny and small group nodules were classified using if-then rules. An SVM with radial basis function (RBF) kernel was used for big nodules classification. Nithila and Kumar [106] computed different textural features from the candidate nodules. First-order statistics (FOS), second-order statistics (SOS), and gray-level co-occurrence matrix (GLCM) have been used as textural features for nodule detection. Chen et al. [117] proposed two new features, viz., “Neighbourhood Feature 1” and “Neighbourhood Feature 2” for false-positive reduction. Feature 1 was computed by taking the ratio between nodule candidate region and the surrounding region. If the value was less than or equal to one, then the candidate nodule will be treated as FP. Retico et al. [118] computed Gradient and Hessian matrix-based features for nodule detection. Choi and Choi [48] proposed a new feature, angular histogram of surface normals (AHSN), for nodule detection. In the lung images, a “wall” is a pleuron or part of another big object near the target. Researchers have proposed a wall detection and elimination algorithm with the help of AHSN feature. Authors have reported in their work that an improvement in accuracy has been achieved by applying the “wall” detection algorithm. Zhang et al. [70] have introduced a new feature named as voxel remove rate (VRR) for nodule detection. The 3-D VRR feature was computed along with four gray features, viz., mean value, standard deviation, skewness, and kurtosis, and five shape-based features, viz., effective radius, elongation, compactness, 1st normalized contour sequence moment, and 3rd normalized contour sequence central moments for nodule identification. Jaffar et al. [115] used multi-coordinate histogram of oriented gradients (MCHOG) feature descriptor and Intensity-based statistical features (IBSF) to discriminate nodules from non-nodules. Froz et al. [120] computed mean direction, circular variance, circular standard variation, mean resulting length, kurtosis, and skewness features for nodule detection. Gonga et al. [88] have used 11 sets of features with Fisher’s linear discriminant analysis (FLDA) classifier for false-positive reduction. Taşci and Uğur [123] have calculated 40 features for false-positive reduction. A feature selection method based on the ranking of features has been carried out after feature extraction. Ten classifiers were evaluated by using these features. Finally, 22 features were selected from 40 features by using a generalized linear model regression (GLMR) classifier. Han et al. [121] have reduced false positives by using rule-based filtering operation. Ciompi et al. [133] investigated a new feature, named as Bag-of-Frequencies (BoF). This feature was computed by analyzing the image intensity in a 3-D spherical neighborhood of a voxel of interest. BoF feature has been applied in two aspects: (1) the discrimination of ROI (nodule) with false positives, e.g., vessels and bifurcation points and (2) the characterization of the spiculated nodule. Intensity profiles were sampled in a multi-scale fashion on spherical surfaces from the center of the nodule. After that, a Fourier transform was used on each intensity profile to obtain a spectrum. Finally, “Bag-of Frequencies” (descriptor) have been created from the spectra. Santos et al. [68] used two texture descriptors, viz., Tsallis’s and Shannon’s Q entropy from the initial nodule candidate set for false-positive reduction. Assefa et al. [114] applied a multi-resolution technique for false-positive reduction. The researchers have used one 2-D separable scaling function along with three separable 2-D wavelets, viz., horizontal, vertical, and diagonal for nodule detection. Cascio et al. [67] computed different 2-D and 3-D features for false-positive reduction. Netto et al. [108] used a 3-D distance transformation for false-positive reduction. Finally, different shape- and intensity-based features have been extracted for nodule detection. Soltaninejad et al. [46] have used 2-D stochastic features and 3-D anatomical features for nodule detection and classification. 3-D averaging was used to differentiate between nodule and vessel. Active contour model has been used for nodule contour extraction that also acts as an FP reducer. Finally, detected small nodules were visualized using the surface rendering (SR) technique. Farag et al. [112] applied two geometric feature descriptors, viz., scale-invariant feature transform (SIFT) and local binary patterns (LBP) for false-positive reduction. Suárez-Cuenca et al. [105] have calculated 18 features for nodule classification. Finally, six classifiers, viz., LDA, QDA, ANN, SVM-dot, SVM-poly, and SVM-ANOVA, were combined using five combination methods. It is reported in their work that the majority-vote rule gives the highest performance among other combination methods. Namin et al. [60] used different intensity features combined with morphological features, e.g., volume size, sphericity, elongation, border variation, and effective radius for false-positive reduction. Murphy et al. [119] applied two successive K-nearest neighbor (KNN) classifiers for false-positive reduction.

Classification

This is the final stage for any CADe system. The output of this stage is yes/no decision. In this stage, recognition of lung nodule has been done with the help of different classifiers. Sensitivity, specificity, and accuracy are the three main parameters to measures the performance of the CADe-based system. In Table 1, the performance of different proposed CADe systems has been given. The choice of a good classifier is an important aspect for any CADe system and varies with the design approach of the system. Most of the works have used SVM as a classifier for nodule detection. Vapnik [134] introduced the original SVM algorithm for classification problems. An SVM finds the maximum-margin hyper-plane that divides the data points into different distinct regions. SVM was used by the works of Choi and Choi [48], Sousa et al. [66], Zhang et al. [70], Han et al. [121], Keshani et al. [99], and Wang et al. [124]. SVM with radial basis function (RBF) kernel has been used in Javaid et al. [75], Froz et al. [120], Aresta et al. [103], Mehre et al. [104], Han et al. [121], and Netto et al. [108]. Qiu et al. [42] used SVM with radial basis Gaussian kernel function. Wang et al. [124] used SVM as a 3-D matrix pattern. Ye et al. [86] have used a weighted SVM. Apart from SVM, Nithila and Kumar [106] have used a neural network (NN). Retico et al. [118] applied NN with 12 input, 14 hidden layers, and 1 output unit, trained with the back-propagation learning algorithm for nodule detection. The works of Farahani et al. [107], Taşci and Uğur [123], Suárez-Cuenca et al. [105], and Antonelli et al. [110] can find ensemble of classifiers for nodule detection. Choi et al. [79] presented a genetic program (GP) technique to optimize the classifier parameters. Gong et al. [111] used a random forest-based classifier. Leemput et al. [62] also have used a random forest classifier with 100 decision trees. Messay et al. [81] have used FLD classifier and quadratic classifier. The authors reported a performance comparison study between the two classifiers and finally, FLD classifier was selected for nodule detection. Murphy et al. [119] used a KNN classifier. Retico et al. [125] have used a voxel-based neural approach (VBNA) based classifier for nodule detection.

Deep Learning-Based Approach Towards Lung Nodule Detection

Deep learning algorithms have become a valuable tool in the field of medical imaging, used for lesion detection, characterization, and analysis. This methodology consists of designing layer wise network architecture by keeping in mind the goal of higher classification accuracy. A survey on deep learning on different network architecture with applications in medical imaging can be found in [135]. Wong et al. [136] and Bruijne [137] presented recent developments and challenges of machine learning in medical imaging. Due to large variations and complexity in biomedical images, it is very difficult to recognize medical objects such as lesions and anatomies from the input image. Therefore, instead of searching for good handcraft-based features, emphasis has been given to the automated feature learning approach called “learning from examples.” In this technique, computers will learn the features automatically from the examples given as input. The learning methods have classified as supervised and unsupervised, in the view of machine learning paradigm. Auto-encoders (AEs) and stacked auto-encoders (SAEs) are examples of unsupervised network models because the labeled training data is absent and only the input image has given to the network. On the other hand, CNNs are the example of supervised network model [135]. Labeled training data is mandatory for CNN architecture. CNN and massive training artificial neural network (MTANN) are the two most recent approaches used in deep learning. Both models use pixel values in images directly as input information, instead of features calculated (handcrafted) from segmented ROIs [138]. A good comparison study of these two end-to-end computing techniques, viz., MTNN and CNN, can be found by the work of Tajbakhsh and Suzuki [139] for nodule detection. However, CNN is the most popular reference architecture used in medical image analysis. From Table 2, it is observed that almost all works have used CNN as the reference architecture for lung nodule detection and characterization purpose. Therefore, a precise and brief introduction to CNN in the context of deep learning has been presented in the “CNN as a Deep Learning Tool” section.

Table 2.

Different nodule detection and characterization system using CNN-based deep learning approach in lung CT images

Sl. No.	First author with publication year	Method	Purpose	Performance
1.	Paul et al. (2018) [37]	Pre-trained vgg-s network with merge CNN	Nodule characterization	Accuracy = 76.79%, ROC = 0.87
2.	Liu et al. (2018) [157]	Multi-view multi-scale CNN	Nodule categorization	Overall accuracy = 92.3% (LIDC dataset) and 90.3% (ELCAP dataset)
3.	Yuan et al. (2017) [155]	Pre-trained vgg network followed by CNN-based features fused with handcraft-based features (multi-view multi-scale CNN)	Nodule categorization	Overall accuracy = 93.1% (LIDC dataset) and 93.9% (ELCAP dataset)
4.	Xie et al. (2017) [156]	Applied CNN at decision level	Nodule characterization	AUC of 96.65%, 94.45%, and 81.24%
5.	Silva et al. (2017) [154]	Genetic algorithm-based CNN	Nodule characterization	Sensitivity = 94.66%, specificity = 95.14%, accuracy of 94.78%, and area under the ROC curve of 0.949.
6.	Wang et al. (2017) [153]	Data-driven-based machine learning model using central focused convolutional neural network (CF-CNN)	Nodule segmentation	Dice score = 82.15% (LIDC dataset) and 80.02% (GDGH dataset)
7.	Dou et al. (2016) [152]	Three 3-D CNN applied with fusion technique	Nodule detection	Sensitivity = 94.4% and sensitivity = 92.2% at 8FPs/scan
8.	Shen et al. (2016) [149]	Multi-crop convolutional neural network (MC-CNN)	Nodule characterization	Accuracy = 87.14%, AUC = 0.93, sensitivity = 77%, specificity = 93%
9.	Setio et al. (2016) [150]	Multiple streams of 2-D ConvNets (multi-view architecture)	Nodule detection	Sensitivity = 85.4% with 1 FP/scan and 90.1% with 4 FPs/scan
10.	Ciompi et al. (2015) [151]	2-D convolution-based architecture named as “OverFeat”	Peri-fissural nodules (PFNs) detection	Area under curve (AUC) = 0.868

CNN as a Deep Learning Tool

This type of architecture initially appeared in the works of Hasegawa et al. [140], Lin et al. [141], and Lo et al. [142]. Lin et al. [141] have applied CNN for FP reduction and achieved a cited accuracy. However, owing to limited computational resources, CNN did not receive popularity at that time. In recent years, with the advent of powerful graphics processing unit (GPUs) such as TitanX (8 GB, 12 GB), CNNs have gained popularity, especially in the field of biomedical image processing. CNN is the most dominated architecture in deep learning and acts as a powerful feature extractor and classifier. The convolution operation is the heart of the CNN architecture. The layers adapting the “convolution operation” are called convolutional layer and responsible for detecting local features in all locations of the input image. CNNs are like multi-layer perceptrons (MLPs) but there are two key differences. First, in CNNs, weights in the network are shared and second, a typical CNN may contain several pooling layers, which is absent in MLPs. Therefore, in CNN, fewer weight parameters require as compared with MPL. The detail of convolution operations and working principle of CNN architecture is available in Dumoulin and Visin [143]; Lecture Series and tutorials from Stanford University [144, 145]. Several CNN-based architecture exists in literature, e.g., the LeNet (1998), AlexNet (2012), VGG16/19 (2014), GoogLeNet/Inception, and ResNet. LeNet and AlexNet are examples of shallow network as it consists of two and five layers, respectively. On the other hand, GoogLeNet and ResNet architecture are examples of deep networks. The CNN architecture used for nodule detection and characterization is somehow deep. Tajbakhsh and Suzuki [139] have mentioned that increasing the depth of the architecture with large training dataset gives better performance. Authors have applied five CNNs separately, viz., a shallow CNN (sh-CNN), the LeNet architecture, a relatively deep (rd-CNN), AlexNet, and FT AlexNet over lung dataset for nodule detection and characterization. FT Alexnet with a large training dataset has given the best performance from the rest of the CNNs for lung nodule detection. Apart from detection, ROIs can be segmented using CNN. Jiang et al. [146] address the medical image semantic segmentation problem by applying the modern CNN-based model. Yan et al. [147] performed a comparative study of slice-level 2-D CNN, nodule-level 2-D CNN, and nodule-level 3-D CNN for nodule detection. Tan et al. [148] have addressed the recent progress and challenges in CNN.

Works Based on CNN Architecture

Recently, deep learning approaches are applying widely owing to its higher detection and classification accuracy. In this subsection, we describe different CNN-based works found so far in the period 2009-April 2018 for nodule detection, categorization, and characterization. The works have been categorized based on (1) type of basic convolution operations used to implement network such as 2-D [149-151] or 3-D [152] or both [153], (2) evolutionary technique applied for CNN parameter optimization [154], (3) combining/fusing feature engineering features with automatic extracted CNN-based features [155, 156], and (4) use of pre-trained network combined with CNN [37]. Table 2 shows the performance of different proposed systems.

2-D Convolution-Based CNN

In this approach, a basic 2-D convolution operation has been used to find local features from the entire image. This is mostly used CNN architecture for its simplicity and computational efficiency. Most of the images are two-dimensional and contain intensity values in the form of a matrix. Several 2-D filters/features are automatically learned from the training dataset by this network. Setio et al. [150] used multiple streams of 2-D ConvNets. Initially, three candidate detector algorithms specially designed for solid, sub-solid, and large nodules have combined for nodule candidate detection. Then for each candidate, a set of 2-D patches of size 64 × 64 pixels have been extracted from differently oriented planes. The researchers have used nine views for this purpose. Then multiple 2-D ConvNets were used for feature extraction. Finally, the outputs of different networks have been merged by applying three fusion techniques, viz., committee fusion, late fusion, and mixed fusion. The authors have reported that late fusion technique gives better detection performance in comparison with committee fusion and mixed fusion techniques. Ciompi et al. [151] proposed a 2-D CNN architecture named as “OverFeat” for peri-fissural nodule detection. The architecture used six convolutional layers, with filter size ranges from 7 × 7 to 3 × 3. Next, three view images (axial, coronal, and sagittal) were fed to the “OverFeat” architecture for nodule feature extraction. Finally, researchers have used a two-stage classifier for peri-fissural nodule detection. Shen et al. [149] have designed a multi-crop convolutional neural network (MC-CNN) by introducing a “multi-crop” pooling strategy for automatic lung nodule characterization. Researchers have used a single network that was capable of producing multi-scale features, instead of using multiple CNN. They proposed a multi-crop-based pooling operation strategy. The multi-crop pooling strategy consists of three pooling operations. Input to the first pooling operation was convolutional features R₀, obtained either from the original input image or from the pooled features. R₁ was the center region cropped from R₀, and R₂ was the center region cropped from R₁. After that, R₀ has max pooled twice and R₁ has max pooled one time to generate the pooled feature maps f₀ and f₁, respectively. R₂ serves as f₂ without a pooling operation. The final multi-crop features were computed by concatenating f₀, f1, and f₂. The work has used randomized leaky rectified linear units (RReLU) as an activation function for SGD learning algorithm that minimize the cross-entropy loss. The proposed architecture was also able to perform semantic label prediction by computing two attributes, viz., nodule subtlety and margin, and can estimate nodule diameter with promising results.

3-D Convolution-Based CNN

In this network, the basic convolution operation has been applied in three directions (x, y, z) at the same time. The convolutional filters used are 3-D, and have applied over the input 3-D data for automatic feature extraction. These networks are more expensive in terms of computation efficiency than 2-D-based network. Convolution operations based on 3-D need more calculations and more memory because 3-D matrix is required to store the extracted features in computer memory. The main advantage of 3-D CNN is that it produces multi-view features with the help of 3-D filters. Dou et al. [152] proposed a 3-D CNN-based architecture for lung nodule detection. Researchers have used three 3-D CNNs to encode spatial information and representative features using hierarchical architecture. For the first architecture, a receptive field of size 20 × 20 × 6 has been used, for second architecture, the size was 30 × 30 × 10, and for third architecture, it was 40 × 40 × 26. Finally, the three CNNs have been merged and act as a feature extractor for nodule detection. The proposed architecture was validated by participating in LUng Nodule Analysis 2016 (LUNA16) challenge and achieved a sensitivity of 94.4%, for nodule detection.

2-D-3-D-Based CNN

The 2-D and 3-D convolution operations can be combined and used to design a hybrid network; we named it as 2-D-3-D CNN. The network is capable of capturing both 2-D and 3-D features from the lung nodule image. In general, the 3-D patch branch is capable of learning multi-view features from multiple CT slices whereas the 2-D patch branch learns multi-scale features. Wang et al. [153] developed a central focused convolutional neural network (CF-CNN) learning model by emphasizing nodule segmentation. The architecture consists of two paths, viz., 3-D CNN-based path and a 2-D CNN-based path. This architecture is a good example of patch-based nodule segmentation approach. Variations in the lesion have been captured using both 2-D and 3-D features from the dataset. The voxels close to center of the image patch were more relevant to the target voxel than the voxels close to edges. Owing to this reason, the researchers have proposed a centrally focused pooling operation. This type of pooling operation reserves location information and many features around the patch center that were not captured in conventional max pooling operation. In central pooling operation, the pooling kernel size varies according to the pooling position and non-uniformly distributed on an input image. On the other hand, in traditional max pooling operation, the pooling kernels are of the same size and uniformly distributed. A parametric rectified linear unit (PReLU) has been used as activation function for more effective and promising results than the conventional ReLU. Researchers have achieved a dice score of 82.15% in LIDC dataset.

Evolutionary Algorithm-Based CNN

This type of network uses evolutionary-based algorithms to optimize different network parameters. Silva et al. [154] designed an evolutionary-based CNN for lung nodule characterization. A good pre-processing step was carried out before applying the classifier. Initially, the Otsu algorithm based on PSO has been applied over the dataset. The goal of this step is to divide each nodule into two sub-regions with maximum inter-class variance. Next, the two-dimensional CT slices were resized and used as an individual sample and three databases each having 21,631 slices have been prepared. Finally, three CNNs were used for nodule characterization. Genetic algorithm was used to optimize the CNN hyperparameters in the convolutional layer. Researchers also have chosen an optimum number of neurons in the fully connected layers with the help of GA. The fitness function has been defined in terms of sensitivity, specificity, and accuracy. A higher weight value was assigned to the sensitivity metric so that the network can classify malignant nodules effectively. The researchers have reported a sensitivity of 94.66% for nodule characterization.

Fusion-Based CNN

In this network, handcraft-based features can be combined with the CNN-based extracted features to form the final feature vector. We have named the network, using this approach, as fused-based CNN. Yuan et al. [155] have categorized five types of lung nodule, viz., well-circumscribed, juxtapleural, juxtavascular, pleural tail, and GGO using CNN. Researchers have used a multi-view multi-scale CNN pre-trained with Visual Geometry Group (VGG) network. Different statistical features were extracted from the nodule image. Next, SIFT feature descriptor was computed and encoded in the form of Fisher vector (FV). The Gaussian mixture model has been used to compute the Fisher geometrical features. Finally, all the handcrafted features were fused with CNN extracted features by using multiple kernel learning (MKL). Yuan et al. [155] achieved an overall accuracy of 93.1%. Liu et al. [157] applied almost the same approach for nodule categorization. Xie et al. [156] have fused handcrafted textural and shape-based features with automated deep features (Fuse-TSD) by using deep convolution neural network for nodule characterization (Fig. 9). The fused features were GLCM texture (T), Fourier shape descriptor (S), and deep learning-based features (D). The authors fused the extracted features in the decision level (decision fusion) which was the output of a fully connected layer. The same features were also fused on the feature extractor level (feature fusion), i.e., before feeding to the classifier. Authors have reported that fusion at decision level was given better performance than at the feature level. Therefore, Fuse-TSD can be used as a higher level decision fusion-based classifier over both manual and automated segmentation process.

Fig. 9.

A CNN-based nodule characterization framework that fuses handcraft features with CNN-based features (Xie et al. [156])

Pre-trained CNN

“Transfer Learning” is one most important concepts in deep learning. In this scenario, the CNN is trained on a large and different dataset. The final weights of the CNN were fined tuned and then used for object detection or classification. This is very helpful as the availability of the medical imaging data is limited, so a network trained with different datasets and modality can be used for the task of deep feature extraction. Paul et al. [37] proposed a new CNN architecture (Fig. 10) for lung nodule characterization. Researchers have used a pre-trained VGG network trained with the Imagenet dataset. Three networks, viz., vgg-f, vgg-m, and vgg-s, have been used for this purpose. Here, the f, m, and s stand for fast, medium, and slow and refer to training time. Vgg-s has been chosen as final pre-trained architecture, as it gives higher classification accuracy. Next, 4096 deep features have been extracted by using vgg-s. Next, three CNN architectures have been trained independently by using the cancerous dataset from NLST. Next, “Architecture-3,” a cascaded CNN, has been chosen for its higher classification accuracy and 1024 deep features were extracted. Finally, nodules were characterized by using all the deep features extracted by using “Architecture-3,” fused with 219 radiomics (handcrafted/feature engineering) features for higher classification accuracy. Researchers have reported an accuracy of 76.79% using pre-trained CNN.

Fig. 10.

A CNN-based nodule characterization framework that pre-trained with VGG network (Paul et al. [37])

It has been observed that different researchers have emphasized different aspects, e.g., some of them on nodule detection, some on nodule characterization, and others have used for FP reduction, by using CNN as basic network architecture. Table 2 gives different CNN-based works in reverse chronological order.

Conclusion

Although there are many studies with the same purpose as described in this paper, it may conclude that a definitive methodology does not exist currently, that could be considered efficient in every aspect of lung nodule detection. Failure of different methodologies include in the parenchyma extraction stage, accurate nodule segmentation stage, e.g., segmentation of juxtavascular nodule, and mostly in the false-positive reduction stage. Apart from that, nodule characterization is very important for cancer studies. Different algorithms may be proposed for nodule characterization based on growth rate, which will ultimately help early detection of cancer and its diagnosis. Automatic analysis of lung cancer staging is another vital finding for a patient. Unfortunately, very few works with significant success rates reported in this direction [22]. In summary, other alternatives must be investigated and at the same time, previous works should be revisited so that an increase in 5-year survival rate, (which is at present only 10-15%) may be achieved.