Recent Progress in Computer-Aided Diagnosis of Lung Nodules on Thin-Section CT

Abstract

Computer-aided diagnosis (CAD) provides a computer output as a “second opinion” in order to assist radiologists in the diagnosis of various diseases on medical images. Currently, a significant research effort is being devoted to the detection and characterization of lung nodules in thin-section computed tomography (CT) images, which represents one of the newest direction of CAD development in thoracic imaging. We describe in this article the current status of the development and evaluation of CAD schemes for the detection and characterization of lung nodules in thin-section CT. We also review a number of observer performance studies in which it was attempted to assess the potential clinical usefulness of CAD schemes for nodule detection and characterization in thin-section CT. Whereas current CAD schemes for nodule characterization have achieved high performance levels and would be able to improve radiologists’ performance in the characterization of nodules in thin-section CT, current schemes for nodule detection appear to report many false positives, and, therefore, significant efforts are needed in order further to improve the performance levels of current CAD schemes for nodule detection in thin-section CT.

I. Introduction

Computer-aided diagnosis (CAD) has become one of the major research topics in medical imaging and diagnostic radiology, and has been applied to various medical imaging modalities including computed tomography (CT), magnetic resonance imaging, ultrasound imaging, and nuclear medicine [1–4]. One of the most important applications of CAD is the detection and characterization of lung cancer, because lung cancer is the leading cause of cancer deaths in the U.S. In fact, the total number of deaths caused by lung cancer is greater than that resulting from colon, breast, and prostate cancers combined [5].

Some evidence suggests that early detection of lung cancer may allow for timely therapeutic intervention and thus a favorable prognosis for the patients [6–8]. Therefore, in the 1970s, screening programs for early detection of lung cancer were carried out with chest radiography and cytologic examination of sputum in the U.S. [9–11] as well as in Europe [12]. As the CT imaging techniques have advanced, screening with low-dose CT has been performed in the U.S. [13,14] and Japan [15–17] since early 1990, because CT is more sensitive than chest radiography in the detection of lung cancer. In a screening program with CT, radiologists must read a large number of images, and they are likely to overlook some lung cancers because of either detection error (failure to detect a cancer) or interpretation error (failure to correctly diagnose a detected cancer) [18]. In such a circumstance, a CAD scheme for detection and for characterization of lung nodules would be particularly useful for the reduction of detection errors and interpretation errors, respectively, because a computerized scheme may detect many cancers missed by radiologists [19,20], and a computerized characterization scheme can provide quantitative information such as the likelihood of malignancy to assist radiologists in diagnosing a detected nodule [21].

For nodule detection in chest radiography, CAD schemes have been developed by many investigators [22–35]. The typical performance of current detection schemes in chest radiography is a 70–75% sensitivity with 1.5–3 false positives per image. For nodule characterization in chest radiography, semi-automated [36–39] and automated [40] CAD schemes have also been developed by a number of investigators. The typical area under the receiver operating characteristics curve (Az value) for distinguishing between benign and malignant nodules in chest radiography is approximately 0.85.

Similarly, CAD schemes for nodule detection in thick-section CT images have been developed by many investigators [41–56]. The typical performance of current CAD schemes in thick-section CT is an 80–90% sensitivity with 1–2 false positives per section, which is translated into tens of false positives per CT scan. Aoyama et al. [57] developed an automated scheme for nodule characterization in thick-section CT with an Az value of 0.85 for distinction between benign and malignant nodules. Because of the relatively large section thickness (5–10 mm), the CAD schemes developed for thick-section CT generally detect and characterize nodules on a slice-by-slice basis. These CAD schemes are considered two-dimensional (2D), because most of the key processing steps such as nodule segmentation and feature extraction are performed on 2D section images.

In a thin-section CT scan, the section thickness is small, typically between 1 and 2.5 mm. A thin-section CT scan includes hundreds of sections and requires considerable time and effort in image interpretation by radiologists, which produces an urgent need for the development of CAD schemes for the detection and characterization of lung cancer. A CAD scheme can generally detect and characterize small and possibly curable cancers more reliably in thin-section CT than in thick-section CT, because the partial-volume effect is much lower in thin-section CT than in thick-section CT, and also because three-dimensional (3D) image processing and analysis techniques become applicable in thin-section CT. Therefore, many investigators have attempted since around 2000 to develop CAD schemes for lung cancer in thin-section CT, which represents one of the newest directions of CAD development in thoracic imaging and is the topic of this review article.

We review only publications concerning CAD schemes for lung nodules in thin-section CT that were published in academic journals and were searchable by use of PubMed [58]. To obtain as many relevant publications as possible, we searched PubMed with the key words “computer lung nodule,” and we manually selected all relevant publications from a total of 201 hits, as of September of 2006.

II. Detection of Lung Nodules

A CAD scheme for nodule detection in CT can be broadly divided into two major steps, i.e., an initial nodule identification step and a false-positive reduction step. The purpose of initial nodule identification is to quickly locate suspicious locations in CT images with a high detection sensitivity for true nodules and, as a result, with a large number of false positives. The purpose of false-positive reduction is to remove as many false positives as possible while maintaining a relatively high detection sensitivity for nodules by analyzing the features of initial nodule candidates. We describe the two steps in the two sections that follow.

II.1. Initial Nodule Identification

To detect lung nodules in CT images, one first needs to separate/segment lung regions from other regions, such as muscle, fat, bone, mediastinum, and background outside the body. All subsequent processing steps are then restricted to the inside of the segmented lung regions. All of the papers on nodule detection [59–68] that we reviewed applied a thresholding technique (or its equivalent such as a simple k-mean clustering [62]) to pixel values for segmentation of lung regions, followed by a morphologic revision of some kind to the segmented lung regions in order to include juxtapleural nodules. Lung region segmentation is a relatively straightforward task, which is typically performed on a slice-by-slice basis.

In most CAD schemes, the initial detection technique was applied directly to the original CT images, whereas in some other schemes, a nodule enhancement filter was first employed as a preprocessing step prior to the application of the initial detection. The application of an effective nodule enhancement filter as a preprocessing step would be advantageous for initial detection of nodules, because, without the nodule enhancement, it may be difficult to identify nodules with low-contrast ground-glass opacity or those connected to blood vessels or airway walls. Li et al. [69] developed a selective enhancement filter for simultaneous enhancement of nodules and suppression of other normal anatomic structures such as blood vessels and airway walls, which were the main sources of false positives for nodule detection in CT. Figure 1 shows a maximum intensity projection (MIP) of an original thin-section CT image with a subtle nodule indicated by an arrow, and an MIP of a nodule-enhanced image by use of the selective enhancement filter. It is apparent that the nodule was enhanced significantly and blood vessels were suppressed remarkably in the enhanced image. Therefore, it would be much easier to detect the subtle nodule in the enhanced image than in the original image. For example, Li et al. [69] applied their selective nodule enhancement filter for initial nodule detection in thick-section CT images, with a sensitivity of 93.4% (71/76) and a false-positive rate of 4.2 per section. Their result was superior to the results of the initial detection techniques developed by Lee et al. [48] [78% (71/91) sensitivity, 9.3 false positives per section], and by Gurcan et al. [50] [90% (57/63) sensitivity, 12.9 false positives per section].

Fig. 1.

Maximum intensity projection of a 3D CT original image with a cancer, indicated by an arrow, and a nodule-enhanced image, in which the nodule was enhanced and blood vessels were suppressed substantially.

Paik et al. [63] also employed a nodule enhancement filter for initial nodule detection based on the surface normal overlap. The output of their filter for each voxel is a score proportional to the number of surface normals that pass through a neighborhood of the voxel. The filter can enhance nodules because nodules tend to have certain convex regions on their surface and thus the inward pointing surface normal vectors tend to intersect or nearly intersect within the tissue. Although blood vessels also have convex surfaces, they have a dominant curvarture along a single direction, as opposed to high curvatures in two directions as is common on the surface of nodules. Therefore, the score for blood vessels is generally lower than that for nodules. Paik et al. compared the surface normal overlap with the Hough transform for nodule enhancement, and they found that the surface normal overlap was more robust than the Hough transform for the enhancement of actual nodules which deviated from ideal models of nodule shape.

Bae et al. [59] developed a CAD scheme for detection of nodules in three categories: isolated, juxtapleural, and juxtavascular nodules. They employed a morphologic matching filter to enhance nodules only for juxtavascular nodules. The morphologic filters were spherical in shape, with four different kernel sizes for identifying nodule candidates ranging from 3 mm to 30 mm. Because the morphologic matching filters were isotropic in three dimension, they would also enhance blood vessels to some extent. Nodule enhancement was not used for initial identification of isolated and juxtapleural nodules.

Thresholding is the most common technique for initial nodule identification in thin-section CT, whether nodule enhancement is applied as a preprocessing step or not. A thresholding technique combined with nodule enhancement would achieve a higher performance level for initial nodule identification, compared to techniques without nodule enhancemnet. For instance, Paik et al. [63] reported a sensitivity of 100% with 165 false positives per CT scan by thresholding nodule-enhanced CT images, whereas Zhao et al. [64] reported a sensitivity of 94.4% with 906 false positives per CT scan by thresholding original CT images wihtout nodule enhancement. As described above, Li et al. [69] also achieved a higher performance level for their initial nodule detection steps in thick-section CT compared to other initial detection methods without nodule enhancement [48,50]. In spite of the advantages of nodule enhancement, most CAD schemes to date did not employ nodule enhancement as a preprocessing step, and they attempted to identify initial nodule candidates directly from the original CT images by use of either a single [59,65–67] or a multiple thresholding technique [61,64]. Such thresholding techniques without nodule enhancement would be more likely to miss low-contrast nodules and to report a larger number of false positives compared to those with nodule enhancement.

Instead of thresholding, Ge and Gurcan et al. [50,62] identified initial nodule candidates by use of a weighted k-means clustering segmentation with two output clusters, i.e., a nodule cluster and a background cluster. They first calculated image features for each pixel from both the original image and a median-filtered image, and they used the image features to classify pixels into the nodule and the background clusters. The criterion for classifying a pixel was the ratio of two distances, which measured how far the feature vector of the pixel was to the nodule cluster center and to the background cluster center, respectively. If the ratio was larger than a threshold, then the pixel was assigned to the nodule cluster, otherwise, it was assigned to the background cluster.

The CAD system developed by McCulloch et al. [68] is unique and consisted of two subsystems. The first sub-system further consisted of a multi-stage modeling architecture ranging from the anatomy model (top level), to the shape model (middle level), and to the signal model (bottom level). These detailed mathematical models quantified the application of domain knowledge for the identification and classification of different regions, including nodules, blood vessels, lung parenchyma, and scars, and these models also attempted to explain what the different regions looked like when they were imaged with a CT scanner. The second sub-system was a Bayesian model selection architecture in which the alternative representations of the regions inside lungs competed with one another to determine the most probable model of the underlying data. Regions for which the nodule model provided the highest probability among all models were considered to be suspicious nodule candidates, and others were considered to be non-nodule candidates.

For initial nodule identification, investigators should clearly address two important issues in their papers. The first is the definition of a criterion for determining whether a true nodule is correctly identified. Paik et al. [63] clearly provided such a criterion; they defined a nodule candidate reported by their CAD as a detected true nodule if the distance between the center of the nodule candidate and the measured center of a “true” nodule was smaller than half the measured diameter of the “true” nodule. The second issue is the explict reporting of the performance level for initial nodule identification because it is an important step of the entire CAD system. Among the papers we reviewed, Paik et al. [63] (a sensitivity of 100% with 165 false positives per CT scan) and Zhao et al. [64] (a sensitivity of 94.4% with 906 false positives per CT scan) appropriately reported the performance levels for their initial nodule identification steps. Ge et al. [62] also reported the performance level of initial nodule detection (a sensitivity of 96% with 6.92 false positives per section); however, the number of false positives was reported on the basis of per section rather than per scan, and was not directly comparable to the performance levels of other methods. Ge et al. reported the number of false positives on the basis of sections, probably because their CT scans contained only parts of lungs.

II.2. False-positive reduction

Whereas the initial nodule identification step locates suspicious nodule candidates in CT images and calculates features for each of the nodule candidates, the false-positive reduction step tries to classify the nodule candidates into nodule and false-positive (non-nodule) categories and, subsequently, to remove false positives by analyzing the features of nodule candidates. One of the most frequently employed and simplest classifier is the rule-based classifier. Many investigators [59,63,64,66,67] have used a rule-based classifier to distinguish nodules from false positives. Because the rule-based classifier generally has a clear semantic meaning, it can be readily comprehended or interpreted by human beings. However, rules were generally determined manually and empirically in existing CAD schemes, which leads to tediousness, long design time, and an overtraining effect. Li et al. [70] devised an automated method to minimize the overtraining effect in the rule-based classifier, in particular, when a large number of rules were created. Sometimes, the rule-based classifier was utilized as a first classsifier followed by a second, more sophisticated classifier such as an artificial neural network (ANN) [66]. In such a case, the rule-based classifier was employed in order quickly to remove obvious false positives (outliers) so that their influence on the training of the second classifier was eliminated.

In addition to the rule-based classifiers and ANN, other classifiers have also been employed for reduction of false positives. For these classifiers, a feature selection process is often necessary because not all features are equally useful in removing false positives. It should be noted that, for a rule-based classifier, such a selection of important features is generally performed by the designers of the rules based on subjective visual judgment. Ge et al. [62] first employed a stepwise process to select important features iteratively by adding new features to or removing features from the subset of currently selected features. They then used the selected features as the input of a linear discriminant analysis (LDA) technique for removing false positives. Boroczky et al. [65] employed genetic algorithms to determine automatically the optimal size of the selected feature subset, and to choose the most relevant features from the entire feature set. The subset of selected optimal features was then used for training a support vector machine (SVM) with a radial basis function for distinction between nodules and false positives.

Whereas ANN, LDA, and SVM are often employed in statistical pattern recognition tasks, the model-based semantic network used by Brown et al. [61] is often employed in structural or semantic pattern recognition. Their model attempted to describe anatomic structures, such as lungs, pulmonary vessels, and pathologic structures such as nodules. The model was represented by using a semantic network. In the structural model, each node contained a set of features, and each arc connecting two nodes represented structural relationships (part of, inside, etc.) between anatomic objects. To distinguish between blood vessels and nodules (the major opacities inside the lungs), candidates were matched to either nodule structures or vessel structures by use of fuzzy logic. A confidence score was calculated for each candidate based on the features of the candidates and a fuzzy membership function. Regions matched to the nodule structure in the model with a high confidence score were considered to be nodules, and those matched to the vessel structure with a high confidence score were considered to be blood vessels and were removed.

II.3. Performance Evaluation for Nodule Detection

Table 1 shows the databases, evaluation methods, and performance levels for the nodule detection schemes we reviewed. Among a total of 9 schemes in Table 1, the first five were developed by investigators in academic research institutions, and the last four were developed, in full or in part, by companies including Philips Medical Systems [65], R2 Technologies [66], Siemens Medical Systems [67], and GE Medical Systems [68]. The schemes were represented by the last names of the first authors and were listed in alphabetical order. Whereas the performance level is very important for a CAD scheme, two equally important aspects are the size and nature of the database employed for training and testing the CAD and the method used to evaluate the performance level.

Table 1.

Databases, evaluation methods, and performance levels for various nodule detection schemes in thin-section CT.

Scheme	Database	Evaluation Method	Performance Level
Bae [59,60]	20 patients 164 solid nodules	Unknown	91.2% sensitivity with 6.9 FPs/scan (3–5mm nodules) 97.2% sensitivity with 4.0 FPs/scan (> 5mm)
Brown [61]	29 patients 77 nodules Partial lungs with 20 mm longitudinal coverage	Hold-out	70% sensitivity with 15 FPs/scan (< 3mm) 100% sensitivity with 15 FPs/scan (> 3mm)
Ge [62]	56 patients, 82 scans 116 solid nodules Partial lungs	Leave-one-out	80% sensitivity with 0.34 FPs/section
Paik [63]	8 patients Unknown number of solid nodules	Leave-one-out	80% sensitivity with 1.3 FPs/scan 90% sensitivity with 5.6 FPs/scan
Zhao [64]	8 patients 266 simulated nodules	Unknown	84.2% sensitivity with 5 FPs/scan
Boroczky [65]	25 patients, 38 scans 52 nodules	Hold-out	Sensitivity unchanged Reduction of 56.4% FPs
Lawler [66]	Unknown	Unknown	Unknown
Marten [67]	20 patients 135 metastastic nodules	Hold-out	76.3% sensitivity with 0.55 FPs/scan
McCulloch [68]	50 patients 35 solid and 8 non-solid nodules	Unknown	69.8% sensitivity with 8.3 FPs/scan

For the database, we listed information such as the number of patients, number of CT scans (some patients may have multiple scans), number of nodules, the characteristics of nodules (solid, non-solid, metastatic, and simulated), and the extent of the CT scan (entire or partial lungs), whenever they were provided. It is apparent that the size of the databases empoyed in these studies was quite small due to the difficulty in collection of thin-section CT images in a relatively short period. Most databases included only solid nodules, which are generally easier to detect than non-solid nodules because solid nodules are generally of relatively high contrast, circular shape, and uniform distribution of density. Marten et al. [67] employed metastatic nodules, which would typically be solid and circular. Zhao et al. [64] used simulated nodules instead of actual nodules. McCulloch et al. [68] clearly stated that their database included 8 non-solid nodules. Whereas most schemes employed CT scans of entire lungs, Brown et al. [61] and Ge et al. [62] used CT scans including parts of lungs; this should be taken into account when readers assess the performance levels of these two CAD schemes.

For the evaluation, leave-one-out and hold-out methods were commonly employed. Both methods are almost unbiased if they are employed appropriately; however, they may become biased if they are used inappropriately. Li et al. [71] identified a number of inappropriate ways to use these evaluation methods, and they made recommendations to correct them. Among the 9 papers in Table 1, four did not mention the evaluation methods used; therefore, it was difficult to judge how reliable their performance levels were.

For the performance level, typical detection rates were between 80% and 90% with 5 to 10 false positives per CT scan. Bae et al. [59] and Brown et al. [61] further provided detailed performance levels for nodules in two size classes, and Paik et al. [63] provided two different performance levels for their CAD scheme. In addition, Bae et al. [59] provided detection rates for nodules in three location classes: 97.4% (76/78), 92.3% (48/52), and 94.1% (32/34) for isolated, juxtapleural, and juxtavascular nodules, respectively. Please note that, because Ge et al. [62] employed CT scans with parts of lungs, the number of false postives was measured based on section instead of scan. Finally, because Boroczky et al. [65] attempted to remove false positives reported by a previous CAD scheme, we only knew that their new technique reduced 56.4% of false postives while maintaining the sensitivity unchanged; the final sensitivity and false positive rate are unknown to us.

Many factors can affect the performance levels of CAD schemes for nodule detection in thin-section CT. Among them, the dose level, section thickness, reconstruction interval, and reconstruction algorithm may have significant effects on computerized nodule detection. Bae et al. [60] specifically studied the effect of section thickness and reconstruction interval on the performance level of their CAD scheme. They utilized three combinations of the section thickness and reconstruction interval to reconstruct CT data for 10 patients with lung nodules: thin group, 1 and 1 mm; overlap group, 5 and 1 mm; and thick group, 5 and 5 mm. The sensitivity and number of false positives per scan in their CAD scheme were: thin group, 95.2% and 5.4; overlap group, 94.2 % and 9.7; and thick group, 88.6% and 23.6. Their findings indicated that the performance of nodule detection improved significantly with a smaller section thickness and a smaller reconstruction interval.

Although the performance levels for the above-mentioned CAD schemes were given, readers should be cautious in directly comparing these performance levels because of the difference in a number of aspects, in particular, in the size of the database, the nature and characteristics of nodules, and the evaluation methods. Appropriate comparison may be possible when a common CT nodule database collected by the Lung Image Database Consortium (LIDC) becomes available to the public [72–74].

There are challenges facing all investigators in the development of CAD schemes for lung nodules in thin-sclice CT. The most prominent one may be the lack of a publicly available large database with which investigators can use to develop and evaluate their CAD schemes properly. LIDC plans to release the lung nodule database officially very soon; it will include 400 CT scans and will be very useful for the developers of CAD schemes. In the future, however, an even larger database including thousands of CT scans would be needed. Another important issus is that most current CAD schemes for nodule detection in thin-section CT were trained with solid nodules; therefore, they may fail to detect non-solid nodules with a good performance level. Similarly, it is unclear how well most current CAD schemes would perform specifically for juxtavascular and juxtapleural nodules.

III. Characterization of Lung Nodules

After a nodule is detected, radiologists want to know whether it is benign or malignant. This task for radiologists can be aided by a CAD scheme which provides a score indicating the likelihood of malignancy of a nodule. Aoyama et al. first developed a CAD scheme for nodule characterization in thick-section CT [57] and then transplanted the CAD scheme to thin-section CT by use of 3D image-processing techniques [75]. Their scheme first segmented a nodule automatically by use of a dynamic programming technique. Based on the extracted outline of the nodule, an inside region and an outside region were determined, which accounted for, respectively, the information inside the nodule region and the context information around the nodule. Forty-one and 15 image features based on 2D sectional data and 3D volumetric data, respectively, were determined from quantitative analysis of the nodule outline and of pixel values. Eight features were automatically selected by use of a stepwise feature selection method, and they were input to a linear classifier for distinguishing between benign and malignant nodules. A leave-one-out testing method was employed to evaluate the performance of this CAD scheme based on a total of 244 patients, including 61 with malignant and 183 with benign nodules. The Az value was employed to measure the performance level of this CAD scheme. The CAD scheme yielded an overall Az value of 0.937 (0.919 for nodules with pure ground-glass opacity, 0.852 for nodules with mixed ground-glass opacity, and 0.957 for solid nodules) for distinction between the 61 malignant and 183 benign lung nodules.

McNitt-Gray et al. [76,77] also developed a CAD scheme for distinction between benign and malignant nodules. Their database contained 35 patients, including 19 with malignant and 16 with benign nodules. All of the patients had at least one volumetric scan and may have had up to four scans imaged 45, 90, 180, and 360 seconds after the injection of contrast agent. Their scheme first employed a semi-automated procedure to segment nodule regions. From a seed point identified by a user of the scheme, a region growing algorithm with user-adjustable upper and lower thresholds was utilized to create a nodule region. The segmented nodule region was reviewed, edited, and approved by one of three thoracic radiologists in their team. Each segmented nodule region was then further partitioned into two regions: one containing only a solid portion, and the other containing only a ground-glass portion. For each of the two regions of every nodule, 31 features were calculated, including 12 in attenuation, 5 in size, 4 in shape, and 10 in contrast enhancement. Feature selection was accomplished by a stepwise model selection search by the Akaike Information Criterion, so that the extent of overfitting was reduced during the subsequent classification step. For three feature sets including 31 features extracted from the solid portion, 31 features from the ground-glass portion, and 62 features from both portions, the feature selection method selected 6, 6, and 5 features, respectively. It seemed that features extracted from the ground-glass portion were not very effective for distinguishing between benign and malignant nodules, regardless whether they were used alone or combined with features from the solid portion. Three classifiers, including linear and quadratic discriminant analysis as well as logistic regression, were employed to distinguish between benign and malignant nodules. A leave-one-out method was utilized for evaluating the performnace level of the CAD scheme. It appeared that the logistic regression classifier provided the highest performance level, and its Az value for distinction between benign and malignant nodules was 0.92.

Kawata et al. [78,79] developed a CAD scheme for distinction between benign and malignant nodules. Their scheme first utilized a deformable surface model to extract nodule regions based on an initial surface placed within a nodule. They then extracted from the segmented nodule regions three features, i.e., the attenuation, shape index, and curvedness value. The shape index measures whether a surface is convex or concave, and the curvedness reflects the degree of curvature on a surface. The histograms of the attenuation, shape index, and curvedness value for pixels within a nodule region were obtained, and the scale of each histogram was employed as a feature. A Fisher linear classifier was trained to provide a score for distinction between benign and malignant nodules. A leave-one-out method was utilized for evaluating the performance level of the CAD scheme based on a total of 62 patients, including 35 with malignant nodules and 27 with benign nodules. Each patient was scanned 3 times at three time points: before the injection of a contrast agent and 2 and 4 minutes after the start of contrast enhancement. The Az values, when all three features were used for the three time points, were 0.91±0.04, 0.99±0.01, and 1.0, respectively. The sensitivity and specificity values were 94% and 74% for the CT images scanned before the injection of contrast agent, 100% and 89% two minutes after contrast enhancement, and 100% and 100% four minutes after contrast enhancement. Table 2 shows the databases, evaluation methods, and performance levels for the above three characterization schemes in thin-section CT.

Table 2.

Databases, evaluation methods, and performance levels for various nodule characterization schemes in thin-section CT.

Scheme	Database	Evaluation Method	Performance Level
Aoyama [57,75]	244 patients 61 malignant nodules 183 benign nodules Contrast-enhanced CT	Leave-one-out	Overall Az = 0.937 0.957: solid nodules, 0.852: mixed GGO 0.919: pure GGO
McNitt-Gray [76,77]	35 patients 19 malignant nodules 16 benign nodules Contrast-enhanced CT	Leave-one-out	Overall Az = 0.92
Kawata [78,79]	62 patients 35 malignant nodules 27 benign nodules Contrast-enhanced CT	Leave-one-out	Overall Az = 1.0

IV. Observer Performance Studies

Observer performanc e studies are often conducted to assess the potential clinical usefulness of CAD schemes for assisting radiologists in the detection and diagnosis of lung nodules in thin-section CT. In a large-scale observer performance study, Brown et al. [80] applied their computerized nodule detection system to eight thin-section CT images (22 lung nodules) with limited longitudinal coverage (partial lungs). Their detection scheme achieved a detection sensitivity of 86.4% with 2.64 false positives per scan for the eight partial CT scans. They then exhibited their CAD scheme at the national scientific meeting of the Radiologic Society of North America, and they recruited 202 observers to participate in their observer study. The 202 observers included 39 thoracic radiologists, 95 non-thoracic radiologists, and 68 non-radiologists. Each participant read from one to eight cases in random order, first without and then with CAD output. Observer performance in nodule detection was measured before and after CAD results were shown. Differences in performance for groups of observers before and after CAD were tabulated and analyzed. In a first analysis involving only the first randomly selected case read by all 202 participants, there were statistically significant increases in both the nodule detection rates and the numbers of false-positive detections for all types of observers. There was a significant difference in detection rates between radiologists and nonradiologists before CAD, but after CAD, there was no significant difference in detection rates between these observer types. In a second analysis involving 13 participants who read all eight cases, the mean detection rate increased from 64.0% without CAD output to 81.9% with CAD output, but the mean number of false-positive detections also increased from 0.144 per case without CAD output to 0.173 with CAD output. This study indicated that, although the observers’ performance was improved overall by use of this specific detection CAD scheme, such improvement was not statistically significant. However, this CAD appeared to help reduce the difference in the performance of nodule detection between different observer groups.

Lee et al. [81] conducted an observer performance study to evaluate the potential clinical usefulness of a CAD system for detecting lung nodules in clinical chest CT. They selected 70 CT scans, including 33 without nodules and 37 with 1–6 nodules (4–15.4 mm in diameter). The CAD system (ImageChecker® CT LN-1000) developed by R2 Technology, Inc. (Sunnyvale, CA) was used, a previous version of which was published in 2003 by Lawler et al [66]. The CAD algorithm was designed to detect nodules with diameters of 4–20 mm. Two chest radiologists working with the CAD system detected a total of 78 nodules, which were employed as reference standards. The CAD system detected 60% (47/78) of nodules and produced 1.56 false-positive detections per CT study. In addition, CAD detected eight nodules that were not mentioned in the original clinical radiology reports. Four independent observers then interpreted the 70 studies without and with the CAD system. The detection rates of the four independent observers without CAD were 81%, 85%, 83%, and 83%, respectively. With CAD, their rates were 87%, 85%, 86%, and 85%, respectively. Although all four radiologists improved their detection rates by use of the CAD system, the differences between these two sets of detection rates did not reach statistical significance. The four observers had 0, 0.1, 0.17, and 0.26 false-positive detections per study without CAD output and 0.07, 0.2, 0.23, and 0.39 with CAD output, respectively. This observer performance study indicated that, although the CAD system at the current performance level assisted radiologists in detecting pulmonary nodules in chest CT, there was also a potential increase in the false-positive rates. Technological improvements to the CAD system would increase the sensitivity and specificity in the detection of pulmonary nodules, and thus would reduce these false-positive detections.

Yuan et al. [82] also attempted to evaluate the performance of the CAD system (ImageChecker® CT LN-1000) developed by R2 Technology. They collected 150 consecutive low-dose screening CT examinations with 628 lung nodules. A radiologist and the R2 CAD system independently evaluated the 150 CT scans for nodule detection. The radiologist detected 518 (82%) of 628 nodules without any false positive. The CAD system detected 456 (73%) of 628 nodules with a total of 478 false positives (3.19 false positives per scan). The radiologist missed 110 nodules that were detected only by the CAD system. In six patients, these were the only nodules detected in the examinations; this may change the imaging follow-up protocol. However, how the CAD output affects radiologists’ detection performance is unknown in this study.

F. Li et al. [83] retrospectively evaluated whether a CAD scheme can help radiologists detect peripheral lung cancers missed at low-dose CT. They used seventeen patients (eight men and nine women) with a missed peripheral lung cancer and 10 control subjects (five men and five women) without cancer at low-dose CT. Fourteen radiologists were divided into two groups on the basis of different image display formats: Six radiologists (group 1) reviewed CT scans on a CRT monitor with a multiformat display, and eight radiologists (group 2) reviewed images with a “stacked” cine-mode display. The radiologists, first without and then with the CAD scheme, indicated their confidence level regarding the presence of cancer and the most likely position of a lesion on each CT scan. With the CAD scheme, the average Az value improved from 0.763 to 0.854 for all radiologists (P = .002), from 0.757 to 0.862 for group 1 (P = .04), and from 0.768 to 0.848 for group 2 (P = .01). The average sensitivity in the detection of 17 cancers improved from 52% (124 of 238 observations) to 68% (163 of 238 observations) for all radiologists (P < .001), from 49% (50 of 102 observations) to 71% (72 of 102 observations) for group 1 (P = .02), and from 54% (74 of 136 observations) to 67% (91 of 136 observations) for group 2 (P = .006). Their study indicated that lung cancers missed at low-dose CT were very difficult to detect, and the use of CAD can improve radiologists’ performance in the detection of these subtle cancers.

Whereas the above observer performance studies were conducted for the detection of lung nodules in thin-section CT, F. Li et al. [75] conducted an observer performance study to evaluate whether a CAD scheme can assist radiologists in distinguishing small benign from malignant nodules on thin-section CT. The dataset used in this observer study consisted of 28 primary lung cancers (6–20 mm) and 28 benign nodules. Cancer cases included nodules with pure ground glass, mixed ground glass, and solid opacity. Benign nodules were selected by matching of their size and pattern to the cancers. Consecutive region-of-interest images for each nodule on thin-section CT were displayed for interpretation in stacked mode on a CRT monitor. The images were presented to sixteen radiologists, first without and then with the computer output, for them to indicate their confidence level regarding the malignancy of a nodule. Figure 2 shows the receiver operating characteristic curves obtained by the CAD scheme and those by radiologists without and with CAD aid. The Az value of the CAD scheme alone was 0.831 for distinguishing benign from malignant nodules. The average Az value for the radiologists was improved from 0.785 to 0.853 by a statistically significant level (P=0.016) with the aid of the CAD scheme. The radiologists’ performance level with the CAD scheme was higher than that of the CAD scheme alone (P<0.05), and also higher than that of the radiologists alone. This sutdy indicated that CAD has the potential to improve radiologists’ diagnostic accuracy in distinguishing small benign nodules from malignant ones in thin-section CT.

Fig. 2.

Receiver operating characteristic curves for distinction between benign and malignant nodules on high-resolution CT.

V. Conclusions

This article reviewed publications related to computerized schemes for nodule detection and characterization in thin-section CT. For nodule detection, the current CAD schemes appear to achieve high detection sensitivities, and also tend to report many false-positive detections; there is no evidence to date indicating that the CAD schemes, at their current performance levels, would improve radiologists’ performance in the detection of nodules in thin-section CT. Further improvements to the current CAD schemes in thin-section CT are needed in order accurately to detect nodules, in particular, those with ground glass opacities. For nodule characterization, however, the current CAD schemes have achieved very high performance levels, and an observer performance study indicates that the CAD schemes would be able to improve radiologists’ performance in the characterization and diagnosis of nodules in thin-section CT.