Machine Learning in Computer-aided Diagnosis of the Thorax and Colon in CT: A Survey

SUMMARY

Computer-aided detection (CADe) and diagnosis (CAD) has been a rapidly growing, active area of research in medical imaging. Machine leaning (ML) plays an essential role in CAD, because objects such as lesions and organs may not be represented accurately by a simple equation; thus, medical pattern recognition essentially require “learning from examples.” One of the most popular uses of ML is the classification of objects such as lesion candidates into certain classes (e.g., abnormal or normal, and lesions or non-lesions) based on input features (e.g., contrast and area) obtained from segmented lesion candidates. The task of ML is to determine “optimal” boundaries for separating classes in the multidimensional feature space which is formed by the input features. ML algorithms for classification include linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), multilayer perceptrons, and support vector machines (SVM). Recently, pixel/voxel-based ML (PML) emerged in medical image processing/analysis, which uses pixel/voxel values in images directly, instead of features calculated from segmented lesions, as input information; thus, feature calculation or segmentation is not required. In this paper, ML techniques used in CAD schemes for detection and diagnosis of lung nodules in thoracic CT and for detection of polyps in CT colonography (CTC) are surveyed and reviewed.

1. Introduction

CAD [1, 2] has been a rapidly growing, active area of research in medical imaging. CAD is defined as detection and/or diagnosis made by a radiologist/physician who takes into account the computer output as a “second opinion” [2]. Evidence suggests that CAD can help improve the diagnostic performance of radiologists/physicians in their image interpretations [3–6]. Consequently, many investigators have participated and developed CAD schemes such as those for detection of lung nodules in chest radiographs (also known as chest x-rays; CXRs) [7–10] and in thoracic CT [11–14], those for detection of microcalcifications/masses in mammography [15], breast MRI [16], and breast US [17], and those for detection of polyps in CTC [18–21].

A CADe scheme of lesions in medical images generally consists of two major components: (1) identification of lesion candidates and (2) classification of the identified candidates into lesions or non-lesions. Segmentation of the organ of interest is the first necessary step before the identification of lesion candidates. The development of the first component, the identification of lesion candidates, generally aims at obtaining a high sensitivity level, because the sensitivity lost in this step cannot be recovered in the later step. The second component, the classification of the identified candidates, is very important, because it determines the final performance of a CAD scheme. The development of the second component aims at removing as many non-lesions (i.e., false-positive (FP) detections in the first step) as possible while minimizing the removal of lesions (i.e., true-positive detections in the first step). Minimizing FPs is very important, because a large number of FPs could adversely affect the clinical application of CADe. A large number of FPs is likely to confound the radiologist’s task of image interpretation and thus lower radiologist efficiency. In addition, radiologists may lose their confidence in CADe as a useful tool. The evaluation of the standalone performance of a developed CAD scheme is the last step of CAD development, and the evaluation of radiologists’ performance with the use of the developed CAD scheme is the important last step in CAD research.

ML plays a very important role in a CAD scheme, because tasks on medical images in a CAD scheme require “learning from examples (or data).” Objects in medical images such as lesions and organs may be too complex to be represented accurately by a simple equation. Modeling of such complex objects often requires a number of parameters that have to be determined by examples or data. For example, a lung nodule is generally modeled as a solid sphere, but there are nodules of various shapes and nodules with internal inhomogeneities, such as spiculated nodules and ground-glass nodules. A polyp in the colon is modeled as a bulbous object, but there are also polyps which have a flat shape [22, 23]. Thus, CAD schemes need “learning from examples or data” to determine a number of parameters in a complex model. ML has been used in the second major step of a CAD scheme, i.e., classification of identified lesion candidates into certain classes (e.g., abnormal or normal, lesions or non-lesions, and malignant or benign) based on input features (e.g., contrast, area, and circularity) obtained from segmented lesion candidates (This class of ML is referred to as feature-based ML, or simply as a classifier). The task of ML here is to determine “optimal” boundaries for separating classes in the multidimensional feature space which is formed by the input features.

ML algorithms for classification include LDA, QDA, multilayer perceptron (one of the most popular artificial neural network (ANN) models) [24], and support vector machines [25]. Such ML algorithms have been applied to lung nodule detection in CXR [26] and thoracic CT [12, 27], classification of lung nodules into benign or malignant in CXR [28] and thoracic CT [29], and polyp detection in CTC [18, 30]. Recently, as available computational power has increased dramatically, PML emerged in medical image processing/analysis which uses pixel/voxel values in images directly, instead of features calculated from segmented regions, as input information; thus, feature calculation or segmentation is not required. PML has also been used in the classification of the identified lesion candidates in CAD schemes.

In this paper, ML techniques used in CAD schemes for detection and diagnosis of lung nodules in CT and for detection of polyps in CTC are surveyed and reviewed. Survey papers for CAD in thoracic CT have been published, including one for lung image analysis in CT with emphasis on a comprehensive survey for computer analysis of the lungs [31], one for CAD in thin-section CT [32], one for CAD in CT with emphasis on CAD performance [33], one for CAD in CT with emphasis on performance comparisons with clinical aspects [34], and one for CAD in both thoracic CT and CTC with emphasis on a methodological overview of major steps in CAD schemes [35]. This present paper focuses on surveys and comparisons of ML techniques in CADe and CADx schemes in thoracic CT and CTC.

2. Classes of Classification Techniques in CAD

There are three classes of classification techniques that have been developed and used in CAD schemes: feature-based classifiers (or feature-based ML), PML, and non-ML-based methods that are defined as methods that do not use ML techniques, such as a procedure that uses a geometrical relationship in a non-learning way. Non-ML methods are not surveyed in this paper.

2.1 Feature-based Classifiers

When an ML algorithm is used for classification, it is generally called a classifier. A standard classification approach based on a classifier such as a multilayer perceptron is illustrated in Fig. 1. First, target lesions are segmented by using a segmentation method. Next, features are extracted from the segmented lesions. Features may include morphologic (or shape-based), gray-level-based (including histogram-based), and texture features. Then, extracted features are entered as input to an ML model such as a multilayer perceptron [24]. The ML model is trained with sets of input features and correct class labels. A class label of 1 is assigned to the corresponding output unit when a training sample belongs to a certain class (e.g., class A), and 0 is assigned to the other output units (e.g., classes B, C, etc.). In the case of two-class classification, one output unit instead of two output units is often used with the output value 0 being class A, and 1 being class B. After training, the class of the unit with the maximum value is determined to be the corresponding class to which an unknown sample belongs. For details of feature-based classifiers, refer to one of many textbooks in pattern recognition such as [24, 25, 36, 37].

Fig. 1.

Feature-based ML (feature-based classifier) for classification of a detected and segmented lesion.

There are several important issues to be considered in the design of ML techniques: generalization, over-fitting, curse of dimensionality, training data annotation, and feature selection.

Generalization in ML is the ability of a trained ML model to perform on unseen cases. The generalization performance of the ML model is estimated by using cases in a test database, which is often lower than the performance for training cases. How to design an ML model with a high generalization performance is an important topic, which is closely related to the over-fitting issue. If an ML model is trained with only a small number of cases, the generalization ability will be lower, because the ML model may fit only the training cases. This is known as over-training (or over-fitting”) [38]. Over-fitting occurs when the number of training cases is too small to determine parameters in the ML model sufficiently. For achieving a high generalization performance, a large number of training cases, e.g., 400–800 cases, is generally required for an ANN in a CADs scheme [39]. For detailed information, please refer to the literature such as [37, 40].

How to estimate the generalization performance with a finite number of testing cases is an important topic as well. To estimate the generalization performance better, resampling schemes such as leave-one-out cross validation, N-fold cross validation, and bootstrapping are often employed [40]. The curse of dimensionality [41] is referred to as the following phenomenon: As the dimensionality of the input feature space for a ML model increases subject to the number of input features, the number of training samples required for the ML model increases exponentially. For detailed information, please refer to the literature such as [40]. To avoid the curse of dimensionality, feature selection and/or dimensionality reduction techniques are often utilized.

Annotating training cases is also an important topic, because the annotation is expensive or time-consuming when the number of training cases is large. There are methods for reducing the annotation labor or annotation itself in the general ML field, but the quality of annotation (or determining “gold standard”) is more important in the CAD research area. In order for the study to be clinically meaningful, “gold standard” annotations (or labels) have to be determined by using more reliable/accurate examinations, e.g., the “gold standard” for lung nodule presence in screening CT should be established by using their confirmation in upper-level follow-up examinations such as diagnostic CT or high-resolution CT (HRCT).

Feature selection has long been an active research topic in ML, because it is one of the main factors that determine the performance of a classifier. It avoids the curse of dimensionality by reducing the input dimension to the classifier. In general, many features are extracted from segmented lesions as the classifier input. Not all of the features, however, would be useful for a classifier to distinguish between lesions and non-lesions, because some of them might be highly correlated with each other or redundant; some of them may not be strongly associated with the given classification task. For designing a classifier with high performance, it is crucial to select “effective” features. In the field of CADe research, one of the most popular feature selection methods is a stepwise feature selection based on Wilks’ lambda. The method has been applied in various CADe schemes because of its simplicity [12, 29, 42]. The Wilks’ lambda criterion is good for LDA, but not necessarily for nonlinear classifiers. One of the most widely used deterministic feature selection methods is sequential forward or backward floating selection (SFFS or SBFS) [43]. SBFS was used for selection of input features for ANNs [44, 45]. SFFS was used for feature selection combined with various classifiers such as Naïve Bayes, SVMs, and AdaBoost [46] in different CADe schemes. Recently, Xu and Suzuki proposed SFFS coupled with an SVM for selection of the most relevant features that maximize the area under the receiver-operating-characteristic (ROC) curve (AUC) [47].

2.2. Pixel/voxel-Based Machine Learning (PML)

Recently, as available computational power has increased dramatically, pixel/voxel-based ML (PML) [35] emerged in medical image processing/analysis, which uses pixel/voxel values in images directly instead of features calculated from segmented regions as input information; thus, feature calculation or segmentation is not required. Because PML can avoid errors caused by inaccurate feature calculation and segmentation, the performance of PML can potentially be better for subtle/complex lesions than that of common feature-based classifiers.

There are three classes of PMLs: neural filters [48–50] (including neural edge enhancers [51, 52]), convolution neural networks (NNs) [53–57] (including shift-invariant NNs [58, 59]), and massive-training artificial neural networks (MTANNs) [19, 60–63] (including multiple MTANNs [12, 49, 50, 60, 64, 65], a mixture of expert MTANNs [20, 66], a multi-resolution MTANN [61], a Laplacian eigenfunction MTANN (LAP-MTANN) [67], a massive-training support vector regression (MTSVR), and a massive-training Gaussian process regression [68]). For details of the architectures and applications of PMLs in medical imaging, refer to a survey paper on PMLs [35].

By extending the neural filter and the neural edge enhancer, two-dimensional (2D) MTANNs [11], which are a class of a PML based on an ANN regression model, have been developed for accommodating the task of distinguishing a specific opacity from other opacities in medical images. The MTANN learns the relationship between input images and corresponding “teaching” images (i.e., ideal or desired images) to distinguish lesions from non-lesions (i.e., FPs). The MTANN is trained with a massive number of subregions/subvolumes extracted from input images together with teaching pixels; hence the term “massive training”. The architecture of an MTANN is shown in Fig. 2. A 2D MTANN consists of a linear-output multilayer ANN regression model, which is capable of operating on voxel data directly [52],[51]. The MTANN is trained with input images/volumes and the corresponding “teaching” images/volumes for enhancement of a specific pattern and suppression of other patterns. The input to the MTANN consists of voxel values in a sub-region/volume (local window or patch) extracted from an input image/volume. The class of MTANNs has been used for classification, such as FP reduction in CAD schemes for detection of lung nodules in CXR [7] and CT [5, 11, 12], distinction between benign and malignant lung nodules in CT [65], and FP reduction in a CAD scheme for polyp detection in CTC [19, 20, 66–68]. The MTANNs have also been applied to pattern enhancement and suppression such as separation of bones from soft tissue in CXR [61, 62], and enhancement of lung nodules in CT [63].

Fig. 2.

Architecture of an MTANN (a class of PML) consisting of an ML regression model (e.g., linear-output ANN regression and support-vector regression) with sub-region (local window or patch) input and single-pixel output.

3. Classification in CAD of the Thorax

Lung cancer continues to rank as the leading cause of cancer deaths in America and other nations such as Japan. The number of lung cancer deaths in each year is greater than the combined number of breast, colon, and prostate cancer deaths in the United States [69]. Because CT is more sensitive than CXR in the detection of small nodules and of lung carcinoma at an early stage [70–72], lung cancer screening programs are being investigated in the United States [73], Japan [70, 71], and other countries with low-dose (LD) helical CT as the screening modality. Evidence suggests that early detection of lung cancer may allow more timely therapeutic intervention and thus a more favorable prognosis for the patient [71, 74]. Helical CT, however, generates a large number of images that must be read by radiologists/physicians. This may lead to “information overload” for the radiologists/physicians. Furthermore, radiologists/physicians may miss some cancers during interpretation of CT images. Therefore, a CAD scheme for detection of lung nodules in LDCT images has been investigated as a useful tool for lung cancer screening.

Classification is a major component in CAD schemes for detection and diagnosis of lung nodules in CT. CAD schemes for detection of lung nodules in thoracic CT (i.e., CADe) generally consists of two major steps: (1) identification of nodule candidates, followed by (2) classification of the identified nodule candidates into nodules or non-nodules (i.e., normal anatomic structures). The second major step in a CADe scheme aims at classification of the nodule candidates identified in the first step into nodules or non-nodules, whereas a CAD scheme for diagnosis (often abbreviated as CADx) aims at classification of the detected nodules (either by a computer or by a radiologist) into benign or malignant nodules.

3.1. Detection of lung nodules

Technical developments of the classification step in CADe schemes for detection of lung nodules in CT are summarized in Table 1. In 1994, Giger et al. [75] developed a CADe scheme for detection of lung nodules in CT. In 1999, Armato et al. [13, 27] extended the method to include 3D feature analysis, a rule-based scheme, and LDA for classification. They evaluated the performance of their scheme with a leave-one-out cross-validation (LOO) test. Kanazawa et al. [76] employed a rule-based scheme with features for classification in their CADe scheme. Gurcan et al. [77] employed a rule-based scheme based on 2D and 3D features, followed by LDA for classification. Lee et al. [78] employed a simpler approach which is a rule-based scheme based on 13 features for classification. Suzuki et al. [60] developed a PML technique called an MTANN for reduction of a single source of FPs and a multiple MTANN scheme for reduction of multiple sources of FPs that had not been removed by LDA. This MTANN approach did not require a large number of training cases: the MTANN was able to be trained with 10 positive and 10 negative cases [79–81], whereas feature-based classifiers generally require 400–800 training cases [79–81]. Arimura et al. [12] employed a rule-based scheme followed by LDA or by the MTANN [60] for classification. Farag et al. [82] developed a template-modeling approach that uses level sets for classification. Ge et al. [83] incorporated 3D gradient field descriptors and ellipsoid features in LDA for classification. Matsumoto et al. [84] employed LDA with 8 features for classification. Yuan et al. [85] tested a commercially available CADe system (ImageChecker CT, R2 Technology, CA). Bi et al. [86] developed an asymmetric cascade of classifiers for classification. Pu et al. [87] developed a scoring method based on the similarity distance of medial axis-like shapes for classification. Retico et al. [88] used the MTANN approach (i.e., a PML technique) for classification. Ye et al. [89] used a rule-based scheme followed by a weighted SVM for classification. Golosio et al. [90] used a fixed-topology ANN for classification, and they evaluated their CADe scheme with a publicly available database from the Lung Image Database Consortium (LIDC) [91]. Murphy et al. [92] used a k-nearest-neighbor classifier for classification. Tan et al. [93] developed a feature-selective classifier based on a genetic algorithm and ANNs for classification. Messay et al. [94] developed a sequential forward selection process for selecting the optimum features for LDA and QDA. Riccard et al. [95] used a heuristic approach based on geometric features, followed by an SVM for classification. Other than the development of CADe schemes, Rao et al. [96] performed an observer performance study with a CADe scheme. Thus, various approaches have been proposed for the classification component in CADe schemes. There are large variations in the performance of CADe schemes: sensitivities ranged from 70–94% with 0.7–64.1 FPs per case. It is difficult to say which CADe scheme performs better because of different databases and testing methods used, without a direct comparison. Some studies used thick-slice CT, and others used thin-slice CT. Some studies used nodules missed by radiologists, and some used nodules detected by radiologists. Evaluation of a CAD scheme with missed cases would be desirable, because the CAD scheme is likely to help radiologists more with such cases. Some studies used screening CT, some used diagnostic CT, and some used HRCT. Testing with screening CT cases would be more appropriate, given the purpose of CADe schemes. Some studies used an LOO test, some used an independent test, and some used N-fold cross-validation. Each testing method has its own advantages and limitations. For detailed information, please refer to the literature [39, 40, 97]. Since the current sensitivity and FP rate of CADe schemes are not high enough compared to radiologists’ performance, further developments of techniques to improve the performance would be necessary. In addition, more studies on the proof of the usefulness of CADe such as observer performance studies and clinical trials would be beneficial in the field.

Table 1.

Classifications component in CADe schemes for detection of lung nodules in CT.

Study	Database	Classifier/Method	Performance
Giger et al. [75]	Thick-slice diagnostic CT scans of 8 patients with 47 nodules	Comparison of geometric features	Sensitivity of 94% with 1.25 FPs per case
Armato et al. [13, 27]	Thick-slice (10 mm) diagnostic CT scans of 43 patients with 171 nodules	Rule-based scheme and LDA with 9 2D and 3D features	Sensitivity of 70% with 42.2 FPs per case in an LOO test
Kanaza wa et al. [76]	Thick-slice (10 mm) screening CT scans of 450 patients with 230 nodules	Rule-based scheme	Sensitivity of 90%
Gurcan et al. [77]	Thick-slice (2.5–5 mm) diagnostic CT scans of 34 patients with 63 nodules	Rule-based scheme and LDA with 6 2D and 3D features	Sensitivity of 84% with 74.4 FPs per case in an LOO test
Lee et al. [78]	Thick-slice (10 mm) diagnostic CT scans of 20 patients with 98 nodules	Rule-based scheme and LDA with 13 features	Sensitivity of 72% with 30.6 FPs per case
Suzuki et al. [60]	Thick-slice (10 mm) screening LDCT scans of 63 patients with 71 nodules with solid, part-solid and non-solid patterns, including 66 cancers	Multiple MTANNs with pixel values in a 9×9 subregion (local window or patch) as input	Sensitivity of 80.3% with 4.8 FPs per case in a validation test
Arimura et al. [12]	106 thick-slice (10 mm) screening LDCT scans of 73 patients with 109 cancers with solid, part-solid and non-solid patterns	Rule-based scheme followed by multiple MTANNs with pixel values in a 9×9 subregion as input (or LDA with Wilks’ lambda stepwise feature selection)	Sensitivity of 83% with 5.8 FPs per case in a validation test (or an LOO test for LDA)
Farag et al. [82]	Thin-slice (2.5 mm) screening LDCT scans of 16 patients with 119 nodules and 34 normal patients	Template modeling approach using level sets	Sensitivity of 93.3% with an FP rate of 3.4%
Ge et al. [83]	82 thin-slice (1.0–2.5 mm) CT scans of 56 patients with 116 solid nodules	LDA with Wilks’ lambda stepwise feature selection from 44 features	Sensitivity of 80% with 14.7 FPs per case in an LOO test
Matsumoto et al. [84]	Thick-slice (5 or 7 mm) diagnostic CT scans of 5 patients (4 of which used contrast media) with 50 nodules	LDA with 8 features	Sensitivity of 90% with 64.1 FPs per case in an LOO test
Yuan et al. [85]	Thin-slice (1.25 mm) CT scans of 150 patients with 628 nodules	ImageChecker CT LN-1000 by R2 Technology	Sensitivity of 73% with 3.2 FPs per case in an independent test
Bi et al. [86]	HRCT scans of 86 patients with 48 nodules	Asymmetric cascade of classifiers with column generation boosting feature selection	Sensitivity of 88% with 0.7 FPs per case in a validation test
Pu et al. [87]	Thin-slice (2.5 mm) screening CT scans of 52 patients with 184 nodules including 16 non-solid nodules	Scoring method based on the similarity distance combined with a marching cube algorithm	Sensitivity of 81.5% with 6.5 FPs per case
Retico et al. [88]	Thin-slice (1 mm) screening CT scans of 39 patients with 102 nodules	Voxel-based neural approach (MTANN) with pixel values in a subvolume as input	Sensitivities of 80–85% with 10–13 FPs per case
Ye et al. [89]	Thin-slice (1 mm) screening CT scans of 54 patients with 118 nodules including 17 non-solid nodules	Rule-based scheme followed by a weighted SVM with 15 features	Sensitivity of 90.2% with 8.2 FPs per case in an independent test
Golosio et al. [90]	Thin-slice (1.5–3.0 mm) CT scans of 83 patients with 148 nodules that one radiologist detected from LIDC database	Fixed-topology ANN with 42 features from multithreshold ROI	Sensitivity of 79% with 4 FPs per case in an independent test
Murphy et al. [92]	Thin-slice screening CT scans of 813 patients with 1,525 nodules	k-nearest-neighbor classifier with features selected from 135 features	Sensitivity of 80 with 4.2 FPs per case in an independent test
Tan et al. [93]	Thin-slice CT scans of 125 patients with 80 nodules that 4 radiologists agreed from LIDC database	Feature-selective classifier based on a genetic algorithm and ANNs with 45 initial features	Sensitivity of 87.5% with 4 FPs per case in an independent test
Messay et al. [94]	Thin-slice CT scans of 84 patients with 143 nodules from LIDC database	LDA and QDA with feature selection from 245 features	Sensitivity of 83% with 3 FPs per case in a 7-fold cross-validation test
Riccardi et al. [95]	Thin-slice CT scans of 154 patients with 117 nodules that 4 radiologists agreed on from LIDC database	Heuristic approach and SVM with maximum-intensity projection data from a volume of interest	Sensitivity of 71% with 6.5 FPs per case in a 2-fold cross-validation test

3.2 Diagnosis of lung nodules

Although CT has been shown to be sensitive to the detection lung nodules, it may be difficult for radiologists to distinguish between benign and malignant nodules on LDCT images. In a screening program with LDCT in New York, 88% (206/233) of suspicious lesions were found to be benign on follow-up examinations [72]. In a screening program in Japan, only 83 (10%) among 819 scans with suspicious lesions were diagnosed to be cancer cases [98]. According to recent findings at the Mayo Clinic, 2,792 (98.6%) of 2,832 nodules detected by a multidetetor CT were benign, and 40 (1.4%) nodules were malignant [99]. Thus, a large number of benign nodules were found with CT; follow-up examinations such as HRCT and/or biopsy were performed on these patients. Therefore, CADx schemes for distinction between benign and malignant nodules in LDCT would be useful for reducing the number of “unnecessary” follow-up examinations.

A number of researchers developed CADx schemes for this task, which distinguish malignant nodules from benign nodules automatically and/or determine the likelihood of malignancy for the detected nodules. The performance of the schemes was generally evaluated by means of ROC analysis [100], because this task is a two-class classification. The AUC [101] was often used as a performance index. Studies on the development of CADx schemes for distinction between malignant and benign lung nodules in CT are summarized in Table 2. In 1998, Kawata et al. [102] described the calculation of nodule features for the purpose of distinction between malignant and benign nodules. In 1999, McNitt-Gray et al. [103] developed a classification scheme based on LDA for distinction between malignant and benign nodules in HRCT. They achieved a correct classification rate of 90.3% for a database of 17 malignant and 14 benign nodules. Matsuki et al. [104] used an ANN with subjective features determined by radiologists for classification between 99 malignant and 56 benign nodules in HRCT and achieved an AUC value of 0.951. Aoyama et al. [42] used LDA for distinction between malignant and benign nodules in thick-slice screening LDCT. They achieved an AUC value of 0.846 for a database of 73 patients with 76 primary cancers and 342 patients with 413 benign nodules. Mori et al. [105] developed a classification scheme for distinction between malignant and benign nodules in contrast-enhanced (CE) CT by using LDA with 3 features (i.e., attenuation, shape index, and curvedness value). Shah et al. [106] employed different classifiers such as logistic regression and QDA with features selected from a group of 31 by using stepwise feature selection based on the Akaike information criterion. Their scheme with logistic regression achieved an AUC value of 0.92 in the distinction between 19 malignant and 16 benign nodules in thin-slice CE-CT. Suzuki et al. [65] developed a PML technique called a multiple MTANN scheme for the classification task. They achieved an AUC value of 0.88 for thick-slice screening LDCT scans of 73 patients with 76 primary cancers and 342 patients with 413 benign nodules. Iwano et al. [107] achieved a sensitivity of 76.9% and a specificity of 80% with their scheme based on LDA with 2 features in their evaluation of HRCT images of 52 malignant and 55 benign nodules. Way et al. [108] incorporated nodule surface features into their classification based on LDA or an SVM, and they achieved an AUC value of 0.857 in the classification of 124 malignant and 132 benign nodules in 152 patients. Chen et al. [109] employed an ANN ensemble to classify 19 malignant and 13 benign nodules, and they achieved an AUC value of 0.915. Lee et al. [110] developed a two-step supervised learning scheme combining a genetic algorithm with a random subspace method, and they achieved an AUC value of 0.889 in the classification between 62 malignant and 63 benign nodules. Other than CADx approaches, Kawata et al. [111] developed a content-based image retrieval approach to provide radiologists with similar images for improving their diagnostic performance in distinction between benign and malignant nodules. Kawata et al. [112] also developed quantitative classification measures that correlate with pathologic characteristics of lung cancer and patients’ prognosis. Thus, various approaches to CADx schemes have been proposed. The database size varied in different studies, from 31–489. Generally achieving high performance for a large database is challenging, because it is likely to contain more variations of nodules. CT scans in the databases included screening LDCT, standard diagnostic CT, and HRCT. Diagnosis of lung nodules on LDCT images would be the most challenging due to a low image quality. Most studies used an LOO test. There are variations in the performance of CADx schemes: AUC values ranged from 0.846–0.951. Once again, it is difficult to say which CADx scheme performs better without a direct comparison. Since the current performance of CADx schemes would be close to or comparable to radiologists’ performance, more studies on the proof of the usefulness of CADx such as observer performance studies and clinical trials would be beneficial in the field.

Table 2.

Classification between malignant and benign nodules (CADx) for thoracic CT.

Study	Database	Classifier/Method	Performance
McNitt-Gray et al. [103]	HRCT scans of 17 malignant and 14 benign nodules	LDA with stepwise feature selection	Correct classification rate of 90.3%
Matsuki et al. [104]	HRCT scans of 99 malignant and 56 benign nodules	ANN with 16 radiologists’ subjective features and 7 clinical data	AUC value of 0.951 in an LOO test
Aoyama et al. [113]	Thick-slice (10 mm) screening LDCT scans of 76 malignant and 413 benign nodules	LDA with Wilks’ lambda stepwise feature selection	AUC of 0.846 in an LOO test
Mori et al. [105]	Thin-slice (2 mm) CE-CT scans of 35 malignant and 27 benign nodules	LDA with 3 features	AUC of 0.91 and 1.0 with non-CE CT and CE-CT, respectively, in an LOO test
Shah et al. [106]	Thin-slice (≤ 3 mm) CE-CT scans of 19 malignant and 16 benign nodules	Logistic regression or QDA with stepwise feature selection from 31 features	AUC of 0.92 with QDA in an LOO test
Suzuki et al. [65]	Thick-slice (10 mm) screening LDCT scans of 76 malignant and 413 benign nodules	Multiple MTANNs with pixel values in a 9×9 subregion as input	AUC of 0.88 in an LOO test
Iwano et al. [107]	HRCT (0.5–1 mm slice) scans of 52 malignant and 55 benign nodules	LDA with 2 features	Sensitivity of 76.9% and a specificity of 80%
Way et al. [108]	CT scans of 124 malignant and 132 benign nodules in 152 patients	LDA or SVM with stepwise feature selection	AUC of 0.857 in an LOO test
Chen et al. [109]	CT (slice thickness of 2.5 or 5 mm) scans of 19 malignant and 13 benign nodules	ANN ensemble with selected features	AUC of 0.915 in an LOO test
Lee et al. [110]	Thick-slice (5 mm) CT scans of 62 malignant and 63 benign nodules	GA-based feature selection and a random subspace method	AUC value of 0.889 in an LOO test

4. Classification in CADe of the Colon

4.1. CADe for detection of polyps in CTC

Colorectal cancer is the second leading cause of cancer deaths in the United States [114]. Evidence suggests that early detection and removal of polyps (i.e., precursors of colorectal cancer) can reduce the incidence of colorectal cancer [115]. CTC, also known as virtual colonoscopy, is a technique for detecting colorectal neoplasms by use of CT scans of the colon. The diagnostic performance of CTC in detecting polyps, however, remains uncertain due to a propensity for perceptual errors in detection of polyps. CADe of polyps has been investigated to address that issue with CTC [116]. CADe has the potential to improve radiologists’ diagnostic performance in the detection of polyps. A number of investigators have developed automated or semi-automated CADe schemes for the detection of polyps in CTC [117–120].

4.2. Classification component in CADe schemes

Major sources of non-polyps (i.e., FPs) remaining after the first step in CADe schemes include haustral folds, residual stool, rectal tubes, the ileocecal valve, and extra-colonic structures such as the small bowel and stomach. Technical developments of the classification step in CADe schemes for detection of polyps in CTC are summarized in Table 3. Many investigators employed feature-based classifiers in the second component of CADe schemes. Gokturk et al. [121] employed an SVM with histogram input that is used as a shape signature for classification. Näppi et al. developed a classification method based on volumetric features [122]. Acar et al. [123] used edge-displacement fields to model the changes in consecutive crosssectional views of CTC data and QDA for classifcation. Jerebko et al. [30] used a multilayer perceptron to classify polyp candidates in their CADe scheme and improved the performance by incorporating a committee of multilayer perceptrons [124] and a committee of SVMs [125]. Wang et al. [126] developed a classification method based on LDA with internal features (geometric, morphologic, and textural) of polyps. Suzuki et al. [127] developed a PML technique called a 3D MTANN by extending the structure of a 2D MTANN [11] to process 3D volume data in CTC. They removed FPs due to rectal tubes by using a single 3D MTANN [127] and multiple sources of FPs by developing and using a mixture of expert 3D MTANNs [20]. Li et al. [128] developed a classification method based on an SVM classifier with wavelet-based features. Wang et al. [129] improved the SVM performance by using nonlinear dimensionality reduction (i.e., a diffusion map and locally linear embedding). Yao at al. [130] employed a topographic height map for calculating features for an SVM classifier.. Suzuki et al. [66] tested a CADe scheme with MTANNs (i.e., a PML technique) on polyps that had been “missed” by radiologists [131] in a multicenter clinical trial [132]. Suzuki et al. [67] also improved the efficiency of the MTANN approach by incorporating principal-component analysis-based and Laplacian eigenmap-based dimension reduction techniques. Xu and Suzuki [68] showed that other nonlinear regression models such as support vector and nonlinear Gaussian process regression models instead of the ANN regression model could be used as the core model in the MTANN framework. Zhou et al. [133] developed projection features for an SVM classifier. Wang et al. [134] improved the performance of a CAD scheme by adding statistical curvature features in multiple-kernel learning. Multiple kernel learning is a recent topic in SVM research.

Table 3.

Classification components in CADe schemes for detection of polyps in CT colonography

Study	Database	Classifier/Met hod	Performance
Gokturk et al. [121]	CTC data (2.5–3.0 mm collimation) of 48 patients in either supine or prone, containing 40 polyps (2–15 mm)	SVM with high-dimensional histograms used as shape signature	Sensitivity of 100% (95%) with a specificity of 0.69 (0.74) [14.3 FPs/patient (12.0 FPs/patient)]
Näppi et al. [122].	CTC data (5 mm collimation) of 40 patients in both supine and prone, including 12 polyps in 11 patients	LDA or QDA with 54 volumetric features (9 statistics of 6 features)	By-patient (by-polyp) sensitivity of 100% (95%) with 2.4 FPs/patient
Acar et al. [123]	CTC data (2.5–3.0 mm collimation) of 48 patients in either supine or prone, containing 40 polyps (2–15 mm)	QDA with edge-displacement fields	Sensitivity of 100% (95%) with a specificity of 0.47 (0.56)
Jerebko et al. [30]	CTC data (5 mm collimation) of 40 patients in both supine and prone, including 29 polyps (3–25 mm) in 20 patients	A multilayer perceptron with 4 features selected from 17 features	Sensitivity of 90% with a specificity of 95% with 32 FPs/patient
Jerebko et al. [124]	CTC data (5 mm collimation) of 40 patients in both supine and prone, including 21 polyps (5–25 mm)	A committee of multilayer perceptrons with 12 features	Sensitivity of 82.9% with a specificity of 95.3% with 5.4 FPs/patient
Jerebko et al. [125]	CTC data (5 mm collimation) of 40 patients in both supine and prone, including 21 polyps (5–25 mm)	A committee of SVMs with 9 selected features	Sensitivity of 86.7% for larger polyps (≥10mm) and 75% for other polyps with 3 FPs/patient in a independent test
Wang et al. [126]	CTC data (5 mm collimation) of 153 patients in both supine and prone, including 61 polyps (4–30 mm) in 45 patients	LDA with internal features of polyps	Sensitivity of 100% and 100% for larger polyps (≥10mm) and other polyps with 4 and 6.9 FPs/patient, respectively
Suzuki et al. [127]	CTC data (1.25–5 mm collimation) of 73 patients in both supine and prone, including 28 polyps (5–25 mm) in 15 patients	Bayesian ANN and a single 3D MTANN with voxel values in a 7×7×7 subvolume as input	By-polyp (by-patient) sensitivity of 96.4% (100%) with 2.1 FPs/patient in an LOO test of the classification part
Suzuki et al. [20]	CTC data (1.25–5 mm collimation) of 73 patients in both supine and prone, including 28 polyps (5–25 mm) in 15 patients	Bayesian ANN and a mixture of expert 3D MTANNs with voxel values in a 7×7×7 subvolume as input	By-polyp (by-patient) sensitivity of 96.4% (100%) with 1.1 FPs/patient in an LOO test of the classification part
Li et al. [128]	CTC data of 44 patients containing 45 polyps (6–9 mm)	SVM classifier with wavelet-based features	Sensitivity of 71% with 5.4 FPs/patient in a 4-fold cross-validation test of the classification part
Wang et al. [129]	CTC data (1.25–2.5 mm collimation) of 791 patients in both supine and prone, including 123 polyps (6–9 mm) and 25 polyps (≥10mm)	SVM with nonlinear dimensionality reduction (i.e., diffusion map and locally linear embedding)	Sensitivity of 83% for polyps (6–9 mm) with 9 FPs/patient
Yao at al. [130]	CTC data (1.25–2.5 mm collimation) of 792 patients in both supine and prone, including 226 polyps (> 6 mm)	SVM classifier with features from a topographic height map	Sensitivity of 93% and 76% for larger polyps (≥10 mm) and other polyps with 1.2 and 3.1 FPs/patient, respectively, in a 10-fold cross-validation test of the classification part
Suzuki et al. [66]	CTC data (1.25–5 mm collimation) of 24 patients in both supine and prone, including 23 polyps (6–25 mm) and a mass (35 mm), that had been “missed” by radiologists in a multicenter clinical trial [132]	Bayesian ANN and a mixture of expert 3D MTANNs with voxel values in a 7×7×7 subvolume as input	By-polyp (by-patient) sensitivity of 96.4% (100%) with 1.1 FPs/patient in an LOO test of the classification part
Zhou et al. [133]	CTC data (1.25–5.0 mm collimation) of 325 patients in supine and/or prone, including 347 polyps and masses (5–60 mm)	SVM classifier with projection features	By-polyp sensitivity of 93.1% and 80.6% for larger polyps (≥10mm) and other polyps with 1.9 and 5.2 FPs/patient, respectively
Wang et al. [134]	CTC data (1.25–2.5 mm collimation) of 66 patients in supine and/or prone, including 96 polyps	Multiple-kernel learning with statistical curvature and 18 geometric features	Sensitivity of 83% with 5 FPs/patient in an LOO test of the classification part

5. Summary

In this paper, ML techniques used in CAD schemes for the thorax and colon have been surveyed. These CAD schemes included CADe and CADx of lung nodules in thoracic CT and CADe of polyps in CTC. The second of the two major components of most CAD schemes, i.e., the classification of lesion candidates, used ML techniques. There are three classes of classification techniques used in CAD schemes: feature-based ML, PML, and non-ML methods. Feature-based ML is the most popular technique in the classification step. Various ML models have been used in this class, including LDA, a multilayer perceptron, an SVM, an ML ensemble, and multiple-kernel learning. Feature selection is an important step for maximizing the performance of a feature-based ML technique, and thus it was often used. The most popular feature selection method in CAD is stepwise feature selection with Wilks’ lambda for linear classifiers such as LDA. Recently, feature selection for nonlinear classifiers has been studied. The most recent development is SFFS under the maximum AUC criterion coupled with an SVM. Recently, PML emerged and used for removal of FPs that had not been removed by feature-based ML. An MTANN is a representative PML model, and there are variations of the MTANNs, including a mixture of expert MTANNs, MTSVR, and Lap-MTANNs. Thus, many investigators have been studying ML in CAD, which indicates the importance of ML in this field. Most CAD schemes employ feature-based MLs that had originally been developed and established in the pattern recognition field. On the other hand, MTANNs were born in the medical imaging field. Evidence demonstrated that PML including MTANNs was effective for improving the performance of CAD schemes. It is hoped that this survey will be useful for researchers in understanding the past studies and the current status of ML in CAD, and in advancing the research area of ML in CAD. It is also hoped to see more original ML techniques/models created in the CAD field.

Biography

Kenji Suzuki received his Ph.D. degree in information engineering from Nagoya University in 2001. From 1993 to 2001, he worked at Hitachi Medical Corporation, and then Aichi Prefectural University as faculty. In 2001, he joined Department of Radiology at University of Chicago. Since 2006, he has been Assistant Professor of Radiology, Medical Physics, and Cancer Research Center there.