Computer-Aided Diagnosis in Medical Imaging: Historical Review, Current Status and Future Potential

Abstract

Computer-aided diagnosis (CAD) has become one of the major research subjects in medical imaging and diagnostic radiology. In this article, the motivation and philosophy for early development of CAD schemes are presented together with the current status and future potential of CAD in a PACS environment. With CAD, radiologists use the computer output as a “second opinion” and make the final decisions. CAD is a concept established by taking into account equally the roles of physicians and computers, whereas automated computer diagnosis is a concept based on computer algorithms only. With CAD, the performance by computers does not have to be comparable to or better than that by physicians, but needs to be complementary to that by physicians. In fact, a large number of CAD systems have been employed for assisting physicians in the early detection of breast cancers on mammograms.

A CAD scheme that makes use of lateral chest images has the potential to improve the overall performance in the detection of lung nodules when combined with another CAD scheme for PA chest images. Because vertebral fractures can be detected reliably by computer on lateral chest radiographs, radiologists’ accuracy in the detection of vertebral fractures would be improved by the use of CAD, and thus early diagnosis of osteoporosis would become possible. In MRA, a CAD system has been developed for assisting radiologists in the detection of intracranial aneurysms. On successive bone scan images, a CAD scheme for detection of interval changes has been developed by use of temporal subtraction images. In the future, many CAD schemes could be assembled as packages and implemented as a part of PACS. For example, the package for chest CAD may include the computerized detection of lung nodules, interstitial opacities, cardiomegaly, vertebral fractures, and interval changes in chest radiographs as well as the computerized classification of benign and malignant nodules and the differential diagnosis of interstitial lung diseases. In order to assist in the differential diagnosis, it would be possible to search for and retrieve images (or lesions) with known pathology, which would be very similar to a new unknown case, from PACS when a reliable and useful method has been developed for quantifying the similarity of a pair of images for visual comparison by radiologists.

1. Introduction

Recently, computer-aided diagnosis (CAD) has become a part of the routine clinical work for detection of breast cancer on mammograms at many screening sites and hospitals (1-25) in the United States. This seems to indicate that CAD is beginning to be applied widely in the detection and differential diagnosis of many different types of abnormalities in medical images obtained in various examinations by use of different imaging modalities. In fact, CAD has become one of the major research subjects in medical imaging and diagnostic radiology (12-25). Although early attempts at computerized analysis of medical images (26-32) were made in the 1960s, serious and systematic investigation on CAD began in the 1980s with a fundamental change in the concept for utilization of the computer output, from automated computer diagnosis to computer-aided diagnosis. In this article, the motivation and philosophy for early development of CAD schemes are presented together with the current status and future potential of CAD in the environment of picture archiving and communication systems (PACS).

2. Historical review of the development of CAD

Large-scale and systematic research and development of various CAD schemes were begun in the early 1980s at the Kurt Rossmann Laboratories for Radiologic Image Research in the Department of Radiology at the University of Chicago. Prior to and since that time, we have been engaged in some of basic research on the effects of digital images on radiologic diagnosis (33-48), and many investigators in the United States and in Europe have become involved in research and development of some aspects of PACS (49). Although PACS would be useful and advantageous in the management of radiologic images in radiology departments and might be beneficial economically to hospitals, it seemed unlikely at that time that PACS would bring a significant clinical benefit to radiologists. Therefore, we thought that a major benefit of digital images must be realized in radiologists’ daily work. Of course, radiologists’ daily work consists of image reading and radiologic diagnosis. Thus, we came to the question “how can radiologists’ diagnosis be helped by the benefits of digital images?” This led immediately to the concept of computer-aided diagnosis.

In the 1980s, the concept of automated diagnosis or automated computer diagnosis was already known from studies (26-31) in the 1960s, but these early attempts were not successful. (The difference and the common aspects between automated diagnosis and computer-aided diagnosis will be discussed in the next section.) Thus, it appeared to be extremely difficult to carry out a computer analysis on lesions involved in medical images. It was, therefore, not easy to predict whether the development of CAD schemes would be successful or fail. Therefore, we thought that we should select research subjects which had the potential to have a major impact in medicine, if CAD could be developed successfully. Some of the most important subjects in medicine at that time were related to cardiovascular diseases, lung cancer, and breast cancer. Therefore, we decided to select three main research projects: for detection and/or quantitative analysis of lesions involved in vascular imaging by Fujita et al. (50) and Hoffmann et al. (51); detection of lung nodules in chest radiographs by Giger et al. (52,53); and detection of clustered microcalcifications in mammograms by Chan et al. (54).

In addition, our efforts on research and development of CAD have been based on three fundamental ideas from the beginning to the present time, as described below. First, our basic strategy for development of methods and techniques for detection and quantitation of lesions in medical images has been based on the understanding of processes that would be involved in image readings by radiologists (20,25). This strategy appeared quite logical and straightforward because radiologists are the ones who have been carrying out very complex and difficult tasks of image reading and radiologic diagnosis. Therefore, we assumed that computer algorithms should be developed based on the understanding of image readings, such as how radiologists can detect certain lesions, why they may miss some abnormalities, and how they can distinguish between benign and malignant lesions.

The second thought was related to the way in which the success of our efforts could be measured if we were successful in the development of CAD. It appeared that the best proof of our success would be the daily use of CAD in routine clinical work at many hospitals around the world. In order for us to realize this, it would be necessary for many industries to commercialize CAD products for the global medical market. Therefore, we decided to produce and protect intellectual properties related to basic technologies of CAD schemes in the form of patents, and to promote these in communicating with many individuals in medical industries for potential efforts toward commercialization of CAD products. Subsequently, our first patent on CAD (55), filed in 1987, has become the most commonly cited patent in the field of CAD technology.

The third thought was to promote the wide acceptance of the CAD concept and to facilitate the global distribution of CAD research by many investigators at many different institutions. Because the success of CAD would require overwhelmingly large efforts on many aspects of CAD research such as the development of computerized schemes for many different types of lesions in many different modalities, observer performance studies, clinical trials, and commercialization, it appeared that we would not be successful if we were the only research group on CAD. Instead, we could be successful if many researchers from many institutions were involved in many aspects of CAD research. Therefore, all researchers on CAD should be considered as promoters and as our colleagues rather than competitors. Therefore, for more than 20 years from the initial phase of CAD research, we have had large scientific exhibits at the Annual Meetings of the Radiological Society of North America (RSNA) which were held in Chicago. In these exhibits, we have presented a comprehensive demonstration of CAD research in chest, breast, and vascular imaging. In 1993, we set up a film digitizer and a computer for real-time demonstration of CAD for detection of clustered microcalcifications on “unknown” new cases of mammograms. We invited in advance 118 breast radiologists to bring their cases to the RSNA meeting for testing of our CAD scheme. The results of this informal validation test were promising (56). From 1996 to 2001, we carried out observer performance studies with a large number of participants during the RSNA meetings for detection of various lesions on chest images without and with the computer output, so that many radiologists would be able to have their own experience of using CAD. The usefulness of CAD was clearly demonstrated (57,58). These demonstrations appeared successful in promoting the CAD concept widely and quickly.

For CAD research on lung cancer, we attempted to develop a computerized scheme for detection of lung nodules on chest radiographs. At that time and still now, it is well known that the visual detection of lung nodules is a difficult task for radiologists, who may miss up to 30% of the nodules. Radiologists have missed these lesions due to the overlap of normal anatomic structures with nodules, i.e., the normal background in chest images tends to camouflage nodules (59-63). Therefore, it was predicted that the normal background structures in chest images would become a large obstacle in the detection of nodules, even by computer. Thus, the first step in the computerized scheme for detection of lung nodules in chest images would be removal or suppression of background structures in chest radiographs. A method for suppressing the background structures is the difference-image technique (64-68), in which the difference between a nodule-enhanced image and a nodule-suppressed image is obtained. This difference-image technique has been useful in enhancing lesions and suppressing the background not only for nodules in chest images (64,65), but also for microcalcifications (54,66) and masses (67) in mammograms, and for nodules in CT (68).

Thus, at the Rossmann Laboratories in the mid 1980s, we had already developed basic schemes for the detection of lung nodules in chest images (52,53) and the detection of clustered microcalcifications in mammograms (54). Although the sensitivities of these schemes for detection of lesions were relatively high even at that time, the number of false positives was very large. It was therefore quite uncertain and unpredictable whether the output of these computerized schemes could be used by radiologists in routine clinical work. For example, the average number of false positives was four per mammogram in the detection of clustered microcalcifications by computer, although the sensitivity was about 85%. However, in order to examine the possibility of practical uses of CAD in clinical situations, we carried out an observer performance study without and with the computer output. To our surprise, radiologists’ performance in detecting clustered microcalcifications was improved significantly when the computer output was available, even with such a large number of false positives. This result was published in 1990 by H.P. Chan et al. (66), as described in detail later.

3. Computer-aided diagnosis and automated computer diagnosis

Early studies on quantitative analysis of medical images by computer (26-31) were reported in the 1960s. At that time, it was generally assumed that computers could replace radiologists in detecting abnormalities, because computers and machines are better at performing certain tasks than are human beings. Thus, the concept of computer diagnosis or automated diagnosis in radiology was established at that time. Although interesting results were reported, these early attempts were not successful, because computers were not sufficiently powerful, advanced image-processing techniques were not available, and digital images were not easily accessible. However, a serious flaw was an excessively high expectation from computers. In fact, many different approaches to automated computer diagnosis have been attempted as aids in decision-making in many fields of medicine since the 1950s. In the medical subspecialty of hematology, for example, R.L. Engle (32), in his review article on 30 years’ experience, stated in his conclusion, “Thus, we do not see much promise in the development of computer programs to simulate the decision making of a physician. ---- However, after many years we have concluded that we should stop trying to make computers act like diagnosticians.”

In the 1980s, however, another approach emerged which assumed that the computer output could be utilized by radiologists, but not replace them. This concept is currently known as computer-aided diagnosis, which has spread widely and quickly. However, in the early phase of research on and development of CAD schemes, some computer scientists criticized CAD by saying that it simply would not work, which has been proved to be completely wrong. The reason for this strong criticism at that time might have been related to an unsuccessful attempt in previous research efforts toward the development of automated computer diagnosis. This seems to indicate that a failure of a specific research project can lead to a strong “incorrect” bias toward another area of research that may look similar to the failed previous research.

However, these two concepts of automated computer diagnosis and computer-aided diagnosis clearly exist even at present. For example, some investigators have been working seriously on the development of automated computer diagnosis. In addition, there are many researchers who believe that the technologies involved in the current CAD schemes could be applied as a means for primary diagnosis, which would be equivalent to automated computer diagnosis. During the panel discussion sessions at two meetings, in 2002, of CARS in Paris and the AAPM in Montreal, about half of the participants voted for the possibility that CAD would be shifted to automated computer diagnosis within 50 years, whereas the other half voted against this prediction. Therefore, it may be useful to understand some of the common features and also the differences between CAD and automated computer diagnosis. The common approach to both CAD and automated computer diagnosis is that digital medical images are analyzed quantitatively by computers. Therefore, the development of computer algorithms is required for both CAD and computer diagnosis.

A major difference between CAD and computer diagnosis is the way in which the computer output is utilized for the diagnosis. With CAD, radiologists use the computer output as a “second opinion,” and radiologists make the final decisions. Therefore, for some clinical cases in which radiologists are confident about their judgments, radiologists may agree with the computer output, or disagree and then disregard the computer. However, for cases in which radiologists are less confident, it is expected that the final decision can be improved by use of the computer output. This improvement is possible, of course, only when the computer result is correct. The higher the performance of the computer, the better the overall effect on the final diagnosis. However, the performance level of the computer does not have to be equal to or higher than that of radiologists. With CAD, the potential gain is due to the synergistic effect obtained by combining the radiologist’s competence and the computer’s capability. Because of these multiplicative benefits, the current CAD has become widely used in practical clinical situations.

With automated computer diagnosis, however, the performance level of the computer output is required to be very high. For example, if the sensitivity for detection of lesions by computer would be lower than the average sensitivity of physicians, it would be difficult to justify the use of automated computer diagnosis. This is because the patients in most advanced countries would not be able to accept a lower level of diagnostic results by computer than the average level achievable by physicians. In addition, the reimbursement for the cost of an automated computer diagnosis may be refused by insurance companies. It would be difficult also to implement automated computer diagnosis if the number of false-positive detections by computer were large, such as one or more false positives per case. In such instances, because every case analyzed by computer would have a chance of involving an abnormality, the physician would be required to verify all of the cases, and this would then not be considered an automated computer diagnosis. Therefore, high sensitivity and high specificity by computer are required for implementing automated computer diagnosis. This requirement is extremely difficult to meet for researchers who are developing computer algorithms for detection of abnormalities on radiologic images. In fact, although CAD has been used widely for detection of breast cancer on mammograms, it would not be possible for most advanced countries to employ the current computer results for automated computer diagnosis.

Another important and distinctive difference between CAD and automated computer diagnosis is the way that the performance and the usefulness would be evaluated quantitatively. The performance level of automated computer diagnosis is equal to the performance achieved by computer. However, the performance level of CAD is equal to the performance achieved by the physician who makes the final decision by using the computer output as a second opinion. Therefore, it is important and essential for CAD that the computer results be utilized effectively by physicians. It is thus necessary to evaluate quantitatively how much physicians can use the computer results to improve their overall performance. Even if the computer results are poor--for example, if both sensitivity and specificity are low--it is possible for physicians to incorporate the computer results efficiently. For example, if the computer could detect subtle lesions which might be difficult for physicians to detect, and also if physicians could disregard “obvious” computer false positives easily, it would be possible to realize CAD. However, if the computer results were poor, automated computer diagnosis could not be implemented.

Therefore, it is important for CAD to assess not only the computer performance, but also the performance by physicians. It is thus necessary to evaluate quantitatively and accurately by use of receiver operating characteristic (ROC) analysis (69-71) whether the performance by physicians can be improved by use of the computer results. In fact, even if the ROC curve for computer results in detecting clustered microcalcifications on mammograms is substantially lower than that by radiologists, the ROC curve obtained by radiologists using the computer results can be improved. The extent of this improvement due to CAD was confirmed to be statistically significant, as reported by H.P. Chan et al. (66) in 1990, thereby providing the first scientific evidence for the benefits of CAD in the detection of lesions. Thereafter, a number of investigators have reported similar positive findings on the usefulness of CAD in detecting various lesions, including clustered microcalcifications (66) and masses (72) in mammograms, lung nodules (73,74) and interstitial opacities (75) in chest radiographs, lung nodules in CT (76), and intracranial aneurysms in MRA (77). Thus they have accelerated research and development on CAD schemes in many different types of examinations for detection of various lesions.

In summary, although there are some common aspects of CAD and automated computer diagnosis, there are also distinct differences between them. Automated computer diagnosis is a concept based on computer algorithms, whereas CAD is a concept that has been established by taking into account equally the roles of physicians and computers. Because of this fundamental difference, the performance levels required for computer algorithms have resulted in a large difference between the two. With automated computer diagnosis, the performance by computers needs to be comparable to or better than that by physicians. With CAD, however, the performance by computers does not have to be comparable to or better than that by physicians, but needs to be complementary to that by physicians. In fact, a large number of CAD systems in the United States and Europe have been employed for assisting physicians in the early detection of breast cancers on mammograms. It should be noted that the overall performance level of these computers is far below the average performance level achievable by physicians, and yet it can be useful in practical clinical situations.

4. Current status of CAD research

The number of papers related to CAD research presented at the RSNA meetings from 2000 to 2005 is listed in Table 1. The majority of these presentations were concerned with three organs - chest, breast, and colon - but other organs such as brain, liver, and skeletal and vascular systems were also subjected to CAD research. It may be noted that the detection of cancer in the breast (1-6), lung (78-84), and colon (85,86) has been or is being subjected to screening examinations (87,88). A large fraction of these examinations give normal results, and the detection of only a small number of suspicious lesions by radiologists is considered both difficult and time-consuming. Therefore, it appears reasonable that the initial phase of practical CAD in clinical situations has begun in these screening examinations. In fact, commercial CAD systems for detection of these cancers are now available for clinical use. Figure 1 shows a comparison of the previous performance level (87% sensitivity at 1.0 false positive per image) in the detection of clustered microcalcifications by computer when the CAD technology was licensed to a company in 1993, with an estimated current performance level (98% sensitivity at 0.25 false positive per image) of the latest commercial CAD system. It is obvious from this result that a substantial improvement was possible due to efforts by people in industry, and thus a successful clinical CAD system has evolved. This may be a good example showing that a small positive seed that germinated at an academic institution can grow substantially by commercialization efforts in industry.

Table 1.

Number of CAD papers related to seven different organs, presented at the Annual Meetings of the Radiological Society of North America (RSNA) in Chicago from 2000 to 2005

Year	2000	2001	2002	2003	2004	2005
Chest	22	37	53	94	70	48
Breast	23	28	32	37	48	49
Colon	4	10	21	17	15	30
Brain	-	4	2	10	9	15
Liver	3	-	5	9	9	9
Skeletal	2	7	7	9	8	5
Vascular etc	5	-	12	15	2	7
Total	59	86	134	191	161	163

Fig. 1.

Comparison of the previous performance level marked by circles (87% sensitivity at 1.0 false positive per image) in the detection of clustered microcalcifications by computer in 1993, when the CAD technology was licensed to a company, with the estimated current performance level, marked by a gray square (98% sensitivity at 0.25 false positive per image), of the latest commercial CAD system.

In mammography, investigators (1-6) have reported results from prospective studies on large numbers of screenees, ranging from 8,682 to 115,571, regarding the effect of CAD on the detection rate of breast cancer, as summarized in Table 2. Although there is a large variation in the results, it is important to note that all of these studies indicated an increase in the detection rates of breast cancer with use of CAD. In addition, Cupples et al. (4) reported a 164% increase in the detection of small (less than 1 cm) invasive cancers, and also a reduction of 5.3 years in the mean age of patients at the time of detection, when CAD was used in mammography.

Table 2.

Prospective studies on the usefulness of CAD for detection of breast cancers in screening mammography

Authors	No. of cases	Gain in cancer detection rate	Increase in recall rate
Freer et al. (1)	12,860	19.5%	18.8%
Gur et al. (2)	115,571	1.7%	0.1%
Birdwell et al. (3)	8,682	7.4%	7.6%
Cupples et al. (4)	27,274	16.1%	8.1%
Morton et al. (5)	18,096	7.6%	10.8%
Dean et al. (6)	9,520	10.8%	26.0%

Because several review articles on CADs for mammography, chest radiography, thoracic CT, CT colonography, and multi-organ, multi-dimensional CAD are included in this Special Issue of the Journal of Computerized Medical Imaging and Graphics, only selected topics with recent findings are discussed below.

4.1 Detection of lung nodules on PA and lateral chest radiographs

In chest radiography and thoracic CT, a number of CAD schemes have been developed, and some commercial systems are available. CAD schemes in chest radiography include the detection and differential diagnosis of lung nodules (89-91) and of interstitial lung diseases (92-96), and the detection of cardiomegaly (97,98), pneumothorax (99), and interval changes (100-105). However, these CAD methods employed only posterior-anterior (PA) views of chest images, even if both PA and lateral views for the same patient were available. In fact, lateral views of chest images were commonly obtained together with PA views in many medical centers and clinical institutions. Even though different opinions have been expressed in the past about the necessity of obtaining PA and lateral views (106-111), it is reasonable to use a lateral view in supporting the performance of a CAD scheme for the detection of lung nodules, and thus in assisting radiologists’ decision-making, if a lateral view together with a PA view has already been obtained.

Recently, Shiraishi et al. (112) developed a computerized scheme for detection of lung nodules in the lateral views of chest radiographs, in order to improve the overall performance in combination with the CAD scheme for PA views. When the CAD scheme was applied only to PA views, the sensitivity in the detection of lung nodules was 70.5%, with 4.9 false positives per image. Although the performance of the computerized scheme for lateral views was relatively low (60.7% sensitivity with 1.7 false positives per image), the overall sensitivity (86.9%) was improved (6.6 false positives per two views), because 20 (16.4%) of the 122 nodules in 106 pairs of PA and lateral views of chest radiographs used in their study were detected only on lateral views. Figure 2 shows relatively large, but very subtle lung nodules located in the right mediastinum region which were not identified by CAD on the PA view, but were correctly identified by CAD on the lateral view. Thus, the CAD scheme by use of lateral-view images has the potential to improve the overall performance for detection of lung nodules on chest radiographs when combined with a conventional CAD scheme for standard PA views.

Fig. 2.

Illustration of a relatively large, but very subtle lung nodule (dotted circles) located in the right mediastinum region which was correctly marked by CAD (triangles) on the lateral view, but was not marked by CAD on the PA view.

4.2 Detection of vertebral fractures on lateral chest radiographs

Vertebral fracture (or vertebral deformity) is a common outcome of osteoporosis, which is one of the major public health concerns in the world. Early detection of vertebral fractures is important because timely pharmacologic intervention can reduce the risk of subsequent additional fractures (113-117). Vertebral fractures can be visible on lateral chest radiographs. However, investigators (118-121) have noted that about 50% of vertebral fractures visible on lateral chest radiographs were underdiagnosed or underreported, even when the fractures were severe. Therefore, Kasai et al. (122,123) developed a computerized method for detection of vertebral fractures on lateral chest radiographs in order to assist radiologists’ image interpretation and thus the early diagnosis of osteoporosis. In the computerized method, a curved search area, which included a number of vertebral end plates, was first extracted automatically, and then was straightened so that vertebral end plates became oriented horizontally. Edge candidates were enhanced by use of a horizontal line-enhancement filter in the straightened image, and a multiple thresholding technique, followed by feature analysis, was used for identification of the vertebral end plates. The height of each vertebra was determined from locations of identified vertebral end plates, and fractured vertebrae were detected by comparison of the measured vertebral height with the expected height.

The cases used in this study were 20 patients with severe vertebral fractures and 118 patients without fractures, as confirmed by the consensus of two radiologists. Preliminary results indicated that the sensitivity of the computerized method for detection of fracture cases was 95% (19/20), with 1.03 (139/135) false-positive fractures per image. For a validation test, the detection accuracy of the computerized method was examined by use of 32 additional fracture cases which were selected independently from training cases. The sensitivity for these cases was 75% (24/32) at 1.03 (33/32) false-positive fractures per image. Figure 3 shows the correct detection of a fractured vertebra on a lateral chest radiograph. Because vertebral fractures can be detected by computer, radiologists can use the detection results as a “second opinion”, and then the detection accuracy of vertebral fractures by radiologists will be improved on lateral chest radiographs, with improvement in the early diagnosis of osteoporosis.

Fig. 3.

Illustration of the correct detection (arrowhead) by computer of a fractured vertebra (dotted circles) below the diaphragm on a lateral chest radiograph, which can be used as a second opinion. Thus, the accuracy of detection of vertebral fractures by radiologists could be improved on lateral chest radiographs, and the early diagnosis of osteoporosis could be improved.

4.3 Detection of intracranial aneurysms in MRA

During the past decade, there has been considerable interest in the roles of “less invasive” imaging modalities such as computed tomographic angiography (CTA) and magnetic resonance angiography (MRA) in the detection of intracranial aneurysms (124-131). However, it is still difficult and time-consuming for radiologists to find small aneurysms, and it may not be easy to detect even medium-sized aneurysms on maximum-intensity-projection (MIP) images, because of overlap with adjacent vessels and because of unusual locations. Therefore, a CAD scheme would be useful in assisting radiologists in the detection of intracranial aneurysms, especially those that are small, by use of MRA. Recently, Arimura et al. (132,133) have developed a computerized scheme for automated detection of unruptured intracranial aneurysms in MRA, based on the use of a 3D selective enhancement filter for dots (aneurysms). The isotropic 3D MRA images were processed by use of a selective, multi-scale enhancement filter (134), as illustrated in Fig. 4. The initial candidates were identified by use of a multiple gray-level thresholding technique on the dot-enhanced images and a region-growing technique with monitoring of some image features. All candidates were classified into four types according to the size and local structures based on the effective diameter and the skeleton image of each candidate, i.e., large candidates and three types of small candidates: a short-branch type, a single-vessel type, and a bifurcation type. In each group, a number of false positives were removed by use of different rules on localized image features related to gray levels and morphology. Linear discriminant analysis was employed for further removal of false positives.

Fig. 4.

The isotropic 3D MRA image in (a) was processed by use of a selective, multi-scale enhancement filter for detection of an intracranial aneurysm (dotted circles), as illustrated in the dot-enhanced image in (b).

The performance of this CAD scheme was evaluated by use of 207 cases with aneurysms which were obtained from three institutions by use of three different types of MR equipment, and thus cases from each institution were tested independently based on a leave-one-out-by-patient test method. The average sensitivity in detecting intracranial aneurysms with sizes ranging from 1mm to 23 mm was 96%, with 3.2 false positives per patient. An observer study was carried out for examining the effect of the computer output on radiologists’ detection performance by use of 22 cases with aneurysms and 28 cases without. The average area under the ROC curve (Az value) for 15 radiologists in the detection of intracranial aneurysms in MRA was improved from 0.931 to 0.983 (P = 0.001) when the computer output was available (77). Therefore, the CAD system would be useful in assisting radiologists in the detection of intracranial aneurysms in MRA.

4.4 Detection of interval changes in successive whole-body bone scans

Bone scintigraphy is the most frequent examination among various diagnostic nuclear-medicine procedures. It is a well-established imaging modality for the diagnosis of osseous metastases and for monitoring of the osseous-tumor response to chemotherapy and radiation therapy. Although the sensitivity of bone scan examinations for detection of bone abnormalities has been considered to be very high, it is time-consuming to identify multiple lesions such as bone metastases of prostate and breast cancers (135,136). In addition, it is very difficult to detect subtle interval changes between two successive abnormal bone scans, because of variations in patient conditions, the accumulation of radioisotopes during each examination, and the image quality of gamma cameras. Therefore, Shiraishi et al. (137) developed a CAD scheme for the detection of interval changes in successive whole-body bone scans by use of a temporal-subtraction image which was obtained with a nonlinear image-warping technique (103). The computerized scheme consisted of several steps: initial image density normalization on each image, image matching for paired images, temporal subtraction by use of the nonlinear image-warping technique, initial detection of interval changes by use of temporal-subtraction images, image feature extraction of candidates of interval changes, rule-based tests by use of 16 image features for removing some false positives, and display of the computer output for identified interval changes.

Based on the consensus of three radiologists, 107 “gold-standard” interval changes were determined among the 58 pairs of successive bone scans in which each scan included both posterior and anterior views. One hundred seven “gold standard” interval changes included 71 hot lesions (i.e., uptake was increased compared with that in the previous scan, or there was new uptake in the current scan) and 36 cold lesions (i.e., uptake was decreased or disappeared) for anterior and posterior views. The overall sensitivity in the detection of interval changes, including both hot and cold lesions, evaluated by use of the resubstitution and the leave-one-case-out methods were 95.3% with 5.97 false positives per view and 83.2% with 6.02, respectively. Figure 5 shows the temporal subtraction image obtained from previous and current bone scan images, and the correct detection of one cold lesion and two hot lesions by computer. Thus, the temporal-subtraction image for successive whole-body bone scans has the potential to enhance the interval changes between two images. Furthermore, the CAD scheme for the detection of interval changes would be useful in assisting radiologists’ interpretation on successive bone scan images.

Fig. 5.

Illustration of temporal subtraction image obtained from previous and current bone scan images. One cold lesion (white solid circle) and two hot lesions (dark dotted circles) on the subtraction image were correctly marked by computer. Thus, the temporal subtraction image for successive whole-body bone scans has the potential to enhance the interval changes between two images.

5. Future potential of CAD in the PACS environment

It is likely that, in the future, some CAD schemes will be included together with other software for image processing in the workstations associated with some specific imaging modalities such as digital mammography, CT, and MRI. However, many other CAD schemes will be assembled as packages and implemented as a part of PACS. For example, the package for chest CAD may include the computerized detection of lung nodules, interstitial opacities, cardiomegaly, vertebral fractures, and interval changes in chest radiographs as well as the computerized classification of benign and malignant nodules and the differential diagnosis of interstitial lung diseases. All of the chest images taken for whatever purpose will be subjected to a computerized search for many different types of abnormalities included in the CAD package, and thus potential sites of lesions together with relevant information such as the likelihood of malignancy and the probability of a certain disease may be displayed on the workstation. For such a package to be used in clinical situations, it is important to reduce the number of false positives as much as possible so that radiologists will not be disturbed by an excessive number of false positives, but will be prompted only or mainly by clinically significant abnormalities.

Radiologists may use this type of CAD package in the workstation in two different ways. One is first to read images without the computer output, and then to request a display of the computer output before making the final decision. If radiologists keep their initial findings in some manner, this mode may prevent a detrimental effect of the computer output on radiologists’ initial diagnosis such as to dismiss incorrectly a subtle lesion because of no computer output, although radiologists were very suspicious of this lesion initially. However, this mode would probably increase the time required for radiologists’ image reading, which is generally undesirable. Another mode is to display the computer output first and then the final decision by the radiologist. With this mode, it is very likely that radiologists can reduce the reading time for image interpretations, but it is uncertain whether they may miss some lesions when no computer output was shown due to computer false negatives. This negative effect can be reduced if the sensitivity in the detection of abnormalities is at a very high level, which may be possible with a package of a number of different, but complementary CAD schemes. For example, although two CAD schemes may miss some lung nodules and other interstitial opacities, it is possible that the temporal subtraction images obtained from the current and previous chest images demonstrate the interval changes clearly, because the temporal-subtraction technique is very sensitive to subtle changes between the two images. This would be one of the potential advantages in packaging of a number of CAD schemes in the PACS environment.

Over the years, investigators have been developing two different types of CAD schemes, one for detection of lesions and another for differential diagnosis of detected lesions based on classification between malignant and benign lesions, and also between different diseases such as different interstitial lung diseases. Although CAD schemes for detection of lesions such as breast lesions on mammograms have been successfully implemented in clinical situations, no serious attempts have been made to apply CAD schemes for differential diagnosis to practical clinical situations, and no commercial systems for CAD for differential diagnosis are available at present. However, the performance levels of CAD schemes for differential diagnosis have been reported to be very high (138-143). For example, a CAD scheme (142,143) for distinction between benign and malignant lung nodules on thoracic CT has been developed based on the determination of the likelihood of malignancy (%) for detected nodules. Feng Li et al. (143) demonstrated the usefulness of this scheme in observer performance studies, in which radiologists’ performance with CAD (Az = 0.853) was greater than that of either radiologists alone (Az = 0.785) or computer output alone (Az = 0.831), with statistically significant differences in Az values. The radiologists generally increased or decreased their confidence level when the likelihood of malignancy was above or below 50%, respectively, and the changes for most nodules tended toward a beneficial effect of the CAD output. Importantly, the correct computer output was able to assist radiologists in improving their decisions on many subtle cases, as illustrated in Fig. 6. For some nodules, however, the radiologists’ initial ratings without CAD were clearly correct; and even when the computer output indicated incorrect results, no serious detrimental effect due to CAD occurred in the radiologists’ ratings, as shown in Fig. 7. Thus, radiologists were able to maintain their own correct judgment when nodules appeared to be obviously benign or malignant despite an incorrect CAD output. Therefore, this study indicated that a synergistic improvement in observers’ interpretation by use of a CAD scheme as a “second opinion” was possible, because the radiologists were able to maintain their own correct opinions on some obvious cases, whereas the computer output assisted in improving their decisions in the majority of subtle cases.

Fig. 6.

Illustration of subtle, difficult nodules in HRCT. The correct computer output for the likelihood of malignancy was able to assist radiologists in improving their decisions, as indicated by a beneficial change in radiologists’ confidence level toward a correct diagnosis for both malignant and benign nodules. The average confidence ratings by 16 radiologists are shown as initial rating and 2nd rating without and with computer output, respectively, where 0 and 1.0 indicate benign and malignant, respectively.

Fig. 7.

Illustration of “obvious” nodules in HRCT, in which radiologists were able to maintain their correct initial ratings for both malignant and benign nodules, even when the computer output indicated incorrect results. Thus, no serious detrimental effect due to CAD occurred in the radiologists’ ratings.

In order to assist radiologists further in their differential diagnosis in addition to the likelihood of malignancy (%), it would be useful to provide a set of benign and malignant images that are similar to an unknown new case in question. If the new case were considered very similar to one or more benign (or malignant) images by a radiologist, he/she would be more confident in deciding that the new case was benign (or malignant). Therefore, similar images may be employed to supplement the computed likelihood of malignancy for implementing CAD for a differential diagnosis. The usefulness of similar images has been demonstrated in an observer performance study (144) in which the Az value in the distinction between benign and malignant nodules in thoracic CT was improved. Figure 8 illustrates the comparison of an unknown case of a mass in a mammogram in the center with two benign masses on the left and two malignant masses on the right. In this simple example, most observers were able to identify the unknown case correctly as being more similar to malignant masses than to benign ones.

Fig. 8.

Comparison of an unknown case of a mass in a mammogram in the center with two benign masses on the left and two malignant masses on the right, which may be retrieved from PACS. Most observers were able to identify the unknown case correctly as being more similar to malignant masses than to benign ones.

There are at least two important issues related to the use of similar images in practical clinical situations. One is the need for a unique database that includes a large number of images which can be used as being similar to those of many unknown new cases, and another is a sensitive tool for finding images similar to an unknown case. Although it may require considerable time and effort, a useful database for this purpose can be developed in the future by use of images stored in PACS. At present, the majority of clinical images in PACS have not been used for clinical purposes, except for images of the same patients, such as in comparisons of a current image with previous images. Therefore, it would not be an overstatement to say that the vast majority of images in PACS are currently “sleeping” and need to be awakened in the future for daily use in many clinical situations. It would be possible to search for and retrieve very similar cases with similar images from PACS, if a reliable and useful method were developed for quantifying the similarity on a pair of images (or lesions) for visual comparison by radiologists. Recent studies indicated that the similarity of a pair of lung nodules in CT (144) and of lesions in mammograms (145,146) may be quantified by a psychophysical measure which can be obtained by use of an artificial neural network trained with the corresponding image features and with subjective ratings given by a group of radiologists. However, further investigations are required for examining the usefulness of this type of new tool for searching really similar images in PACS.

6. Conclusion

Computer-aided diagnosis has become a part of clinical work in the detection of breast cancer by use of mammograms, but is still in the infancy of its full potential for applications to many different types of lesions obtained with various modalities. CAD is a concept based on the equal roles of physician and computer, and thus is distinctly different from automated computer diagnosis. In the future, it is likely that CAD schemes will be incorporated into PACS, and that they will be assembled as a package for detection of lesions and also for differential diagnosis. CAD will be employed as a useful tool for diagnostic examinations in daily clinical work.