Computer-aided diagnosis for the classification of focal liver lesions by use of contrast-enhanced ultrasonography

Abstract

The authors developed a computer-aided diagnostic (CAD) scheme for classifying focal liver lesions (FLLs) as liver metastasis, hemangioma, and three histologic differentiation types of hepatocellular carcinoma (HCC), by use of microflow imaging (MFI) of contrast-enhanced ultrasonography. One hundred and three FLLs obtained from 97 cases used in this study consisted of 26 metastases (15 hyper- and 11 hypovascularity types), 16 hemangiomas (five hyper- and 11 hypovascularity types) and 61 HCCs: 24 well differentiated (w-HCC), 28 moderately differentiated(m-HCC), and nine poorly differentiated (p-HCC). Pathologies of all cases were determined based on biopsy or surgical specimens. Locations and contours of FLLs on contrast-enhanced images were determined manually by an experienced physician. MFI was obtained with contrast-enhanced low-mechanical-index (MI) pulse subtraction imaging at a fixed plane which included a distinctive cross section of the FLL. In MFI, the inflow high signals in the plane, which were due to the vascular patterns and the contrast agent, were accumulated following flash scanning with a high-MI ultrasound exposure. In the initial step of our computerized scheme, a series of the MFI images was extracted from the original cine clip (AVI format). We applied a smoothing filter and time-sequential running average techniques in order to reduce signal noise on the single MFI image and cyclic noise on the sequential MFI images, respectively. A kidney, vessels, and a liver parenchyma region were segmented automatically by use of the last image of a series of MFI images. The authors estimated time-intensity curves for an FLL by use of a series of the temporally averaged MFI images in order to determine temporal features such as estimated replenishment times at early and delayed phases, flow rates, and peak times. In addition, they extracted morphologic and gray-level image features which were determined based on the physicians’ knowledge of the diagnosis of the FLL, such as the size of lesion, vascular patterns, and the presence of hypoechoic regions. They employed a cascade of six independent artificial neural networks (ANNs) by use of extracted temporal and image features for classifying five types of liver diseases. A total of 16 temporal and image features, which were selected from 43 initially extracted features, were used for six different ANNs for making decisions at each decision in the cascade. The ANNs were trained and tested with a leave-one-lesion-out test method. The classification accuracies for the 103 FLLs were 88.5% for metastasis, 93.8% for hemangioma, and 86.9% for all HCCs. In addition, the classification accuracies for histologic differentiation types of HCCs were 79.2% for w-HCC, 50.0% for m-HCC, and 77.8% for p-HCC. The CAD scheme for classifying FLLs by use of the MFI on contrast-enhanced ultrasonography has the potential to improve the diagnostic accuracy in the histologic diagnosis of HCCs and the other liver diseases.

INTRODUCTION

Noninvasive diagnosis of focal liver lesions (FLLs) hasbeen performed based on contrast-enhanced computed tomography¹ (CT) or magnetic resonance imaging (MRI).^{2, 3, 4} Although the performance of ultrasonography for the detection of FLLs was considered to be poor compared with those of CT and MRI,⁵ recent progress in ultrasonography with contrast agents and pulse-inversion⁶ (phase-inversion) imaging techniques allowed real-time assessment of the liver vascularity pattern, and thus, improvement in the diagnostic accuracy for the classification of FLLs.^{6, 7, 8, 9} For example, Quaia et al.⁹ reported that the overall diagnostic accuracy for 452 FLLs by use of conventional baseline ultrasonography(B-mode and Doppler U.S.) was improved by use of contrast-enhanced ultrasonography from 49% to 85% for reader 1 and from 51% to 88% for reader 2. Although the use of some microbubble contrast agents for diagnosing liver disease in ultrasonography is under investigation in the United States, they have been used widely for liver diagnosis in most countries in Europe, Asia, and Canada.^{10, 11}

The diagnosis of liver diseases on ultrasonography with microbubble contrast agents has been studied thoroughly.^{6, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25} For example, benign lesions of hemangioma and focal nodular hyperplasia (FNH) consistently showed sustained enhancement such that the mass appeared equal to or greater in echogenicity than the adjacent liver for the duration of the scanning interval, often as long as 4 min.²² Liver metastasis, which is the most frequent malignant FLL, can be shown as hypovascularity and rim enhancement in contrast-enhanced ultrasonography when the metastases are from breast, lung, and colon cancers.¹⁴

Among various kinds of liver diseases, hepatocellular carcinoma (HCC) is the most common primary liver cancer, which usually occurs as a complication of chronic liver disease and most often arises in a cirrhotic liver.^{26, 27} Because accurate and early diagnosis of HCC is essential for treatment, accurate surveillance of patients with liver cirrhosis is of great clinical importance. In order to diagnose HCCs at their early stages, it is important to determine the degree of histologic differentiation in clinical practice. Vascular invasion is one of the most important determinants of tumor grade and also is reported to be correlated with the degree of histologic differentiation.²⁸ Therefore, the classification of HCCs into three degrees of histologic differentiation, such as well differentiated, moderately differentiated, and poorly differentiated, is important as well as the benign∕malignant classification of FLLs. Although HCCs are generally found to be highly vascular lesions in contrast-enhanced ultrasonography with pulse-inversion imaging,²² it is difficult to classify HCCs according to their degrees of histologic differentiation without the results of pathologic study. Because of the incidence of postoperative complications, severity of postoperative pain, and increase in the number of hospitalized days, it is recommended to use less invasive real-time percutaneous local ablation therapy for the treatment of well differentiated HCC, rather than a partial hepatectomy. Therefore, for performing real-time percutaneous local ablation therapy during ultrasound examination, it would be useful if the histologic differentiation of the HCC were provided to the physician.

The concept and methodology of computer-aided diagnosis (CAD) to assist radiologists in detecting abnormal lesions and improving the classification accuracy of the differential diagnosis have been developed and studied in various radiologic imaging methods.^{29, 30} CAD may be defined generally as a diagnosis made by physicians who take into account the results of automated computer analysis of medical images. The computer output may be used as a “second opinion” for improving physicians’ decision-making and avoiding oversights. Although there were several studies on CAD schemes for the classification of liver lesions by use of CT and MR images,^{31, 32} to the best of our knowledge, there has been no application of a CAD scheme for the classification of FLLs in ultrasonography.

In this study, we developed a CAD scheme for classifying FLLs into liver metastasis, hemangioma, and three histologic differentiation types of HCC, by use of microflow imaging (MFI^™, Toshiba Medical Systems Corp., Japan) in contrast-enhanced ultrasonography.

MATERIAL AND METHODS

Microflow imaging with contrast-enhanced ultrasonography

In contrast-enhanced ultrasonography, it is possible to obtain a series of enhanced blood flow images continuously in a low-power imaging mode. In the flash-replenishment (FR) method,³³ the microbubbles in the scan volume are destroyed by a burst scan at high-mechanical-index (MI) transmission. The microbubbles in the scan volume are then replenished in the low-MI imaging mode. This concept was first used in myocardial perfusion in high-MI imaging mode by Wei et al.,³⁴ and successful quantitative results were demonstrated.

On the other hand, for observing the structure of the minute vessels, the usefulness of a max-hold imaging technique has been demonstrated by Burns³⁵ and by Powers et al.³⁶ Microflow imaging (MFI) is an imaging method combining the FR method and a max-hold imaging technique. Figure 1 is a schematic illustration of the MFI procedure and provides sample images in each step. The scanning plane which included the largest and most conspicuous section of the FLL was determined initially by use of conventional B-mode scans. Then a contrast agent was injected as a 1.5 ml bolus into an antecubital vein with a 21-gauge peripheral intravenous cannula. During the arterial phase of the contrast enhancement, which was timed for 45 s after the injection, the location of the scanning plane and the size of a field of view were adjusted suitably by use of harmonic imaging (commercially named pulse subtraction). The sequence of MFI was started with the patient’s breath-hold and a burst scan with high-MI (1.3–1.6) scanning of five frames. Just after the burst scan, FR low-MI (0.07–0.09) scanning was started simultaneously with a max-hold imaging technique so that the maximum brightness on each pixel was maintained and displayed as persistence vision. In MFI, the inflow high signals obtained with the FR method, which were due to the vascular patterns and the contrast agent replenished from the regions adjacent to the scanning plane, were accumulated until the perfusion of inflow high signals was saturated. In this study, we defined an “early phase” and a “delayed phase” in the MFI as a replenishment time for reaching 50% and 98% of the maximum average pixel value within a FLL, respectively. In addition, MFI was repeated during the extended portal venous phase (from 45 to 70 s after injection), if necessary.

Figure 1.

Illustration of the procedure of MFI and its sample images. Schemas on the left show sections crossing the scanning plane for the MFI. Injected contrast agent (microbubbles) was enhanced by harmonic imaging which depresses the normal structures. Diffused microbubbles in the scanning plane were destroyed by high MI in the first step of MFI, and then the microbubbles in the adjacent region were replenished into the scanning plane corresponding to the vascularity patterns of an FLL. The MFI images were recorded sequentially until the replenishment was saturated.

Image database

We used 97 MFI cases with 103 FLLs in this study. All cases were collected at the Tokyo Medical University Hospital with IRB-approved patients’ consent. The 103 lesions consisted of 26 metastases, 16 hemangiomas, and 61 HCCs: 24 well differentiated (w-HCC), 28 moderately differentiated (m-HCC), and nine poorly differentiated (p-HCC). The 26 metastases and 16 hemangioma lesions were divided into two groups, including hypervascularity (15 metastases, five hemangiomas) and hypovascularity (11 metastases, 11 hemangiomas) types. Pathologies of all cases were determined based on biopsy or surgical specimens. The patient identification information such as patient ID, examination date, gender, and age were blinded to the author so that the patient identification information was secure. Among 97 MFI cases, two cases had multiple lesions of metastases (the numbers of metastases were four and three), and one case had two lesions of HCCs (one w-HCC and one m-HCC) in the same sequential images of ultrasonography. Locations and approximate contours for the 103 FLLs on contrast-enhanced ultrasonography were determined manually by an experienced physician (K.S.). If one patient had two or more MFI studies for the same lesion in the arterial and portal venous phases, only one MFI study, in which the vascular architecture was most clearly visualized, was selected (by K.S.) and included in the database.

All of the image data for the development of the CAD scheme were originally provided with the AVI format cine clip obtained at the time of examination. All cine clip files had the same format, an 800×600 matrix size and 8-bit gray scale, but with various pixel sizes (mean 0.245 mm, range from 0.109 to 0.416 mm) due to the lesion size and the depth of the lesions from the skin surface. The pixel size for each case was calculated by use of the 1 cm scale displayed in the original cine clip and the number of pixels corresponding to this scale. The acoustic frame rate (AFR) for acquiring ultrasonography was 15 frames per second, and the video frame rate (VFR) for recording image data with the AVI format was 15 or 30 frames per second. The average acquisition time for the MFI was 11.9 s (range from 4.1 to 23.1 s), which depended on the perfusion of the target tissue, and the breath-holding time of the patients during the examination.

In this study, all cases were obtained by use of the ultrasound equipment SSA-770A (Aplio^™, Toshiba Medical Systems Corp., Otawara, Japan) with a 3.75 MHz convex transducer (PSK-375BT), with a second generation ultrasound contrast agent, SonoVue^™ (Bracco, Milan, Italy). The imaging mode was wide band harmonic imaging with transmission and reception frequencies of 3.75 and 7.5 MHz, respectively.

Characteristics of FLLs in MFI images

In general, the normal liver is supplied mainly by portal veins, whereas HCCs and hypervascularity metastases are supplied by the hepatic artery as well as tumor vessels. Therefore, from the diagnostic point of view of physicians, enhancement patterns of FLLs in the arterial and portal venous phases of MFI images can be used for characterizing FLLs. For example, the enhancement patterns of the FLL can be described as (1) slow or rapid, (2) uniform or sparse, and (3) strong or weak. In addition, the enhancement patterns of the FLL in the MFI can be distinguished with homogeneous, heterogeneous, centrifugal, and centripetal progressions. The centrifugal and centripetal progressions represent the direction of the enhancement within a FLL such as a change from central to peripheral and another change from peripheral to central, respectively. Figure 2 shows examples of centrifugal and centripetal cases. When two MFI images were compared at the early and the delayed phases, it is apparent that the FLLs were enhanced in two opposite directions, or in no direction. Generally, when only the pulse inversion method was used in the arterial and the portal venous phases of contrast-enhanced ultrasonography, the centripetal progression was typical for hemangioma;²³ however, these patterns were observed occasionally in HCCs on the MFI images.³⁷

Figure 2.

Examples of two FLLs (m-HCCs) which represent centrifugal (upper) and centripetal (lower) progressions of replenishment from early (left) to delayed (right) phases.

The comparison of the replenishment pattern of FLLs with those in adjacent liver parenchyma (ALP) regions was also important for characterizing the FLLs on the MFI. By use of this comparison, the FLLs can be classified in three categories, including hyperechoic, hypoechoic, and isoechoic patterns. Figure 3 shows examples of the three patterns of FLLs. We defined hyperechoic when the echogenicity of the FLL was greater than that of the ALP; hypoechoic when the echogenicity of the FLL was less than that of the ALP; and isoechoic when the echogenicity of the FLL was equal to that of the ALP. The hypoechoic pattern of an FLL in pulse-inversion imaging is shown frequently in cases of hypovascularity metastasis and hemangioma.^{9, 14, 23}

Figure 3.

Examples of three FLLs which represent hyperechoic (upper left: m-HCC), hypoechoic (upper right: metastasis), and isoechoic (lower left: w-HCC) replenishment patterns.

In terms of the diagnostic difficulty for classifying various types FLLs, classifications of hypovascularity lesions including both hemangioma and metastasis were considered to be obvious compared to the classifications of the other hypervascularity lesions because the replenishment pattern of hypovascularity lesions usually indicated a hypoechoic pattern. Among hypervascularity lesions, the classification of hypervascularity hemangioma was relatively obvious because the size of hypervascularity hemangioma was likely to be large compared with the other hypervascularity lesions. The enhancement patterns of w-HCC and p-HCC were likely to be homogeneous and heterogeneous, respectively, compared to that of m-HCC. It has been considered most difficult to distinguish between m-HCC and hyper-vascularity metastasis.

Computerized scheme for classification of FLLs

Figure 4 shows our CAD scheme for the classification of FLLs on contrast-enhanced ultrasonography obtained with MFI. This computerized scheme consists of six major steps, i.e., (1) extraction of a series of MFI images from the original cine clip by application of a smoothing filter and time-sequential running-average techniques; (2) automated segmentations of a kidney, vessel-like patterns, the ALP region, and a central∕peripheral region of a FLL; (3) estimation of time-intensity curves for a FLL by use of a series of temporally averaged MFI images for determination of temporal features such as replenishment times at early and delayed phases, flow rate, and peak pixel value; (4) extraction of morphologic and gray-level image features for a FLL; (5) stepwise feature selections for distinguishing two groups in six different decisions; and (6) application of a cascade of independent six artificial neural networks (ANNs) by use of independently selected features for classifying five types of liver diseases. Details of our computerized scheme are described below.

Figure 4.

Overall computerized scheme used in this CAD scheme for the classification of FLLs on MFI images.

Extraction of the series of MFI images

In the initial step of our computerized scheme, a series of input images of the MFI was cropped automatically with a 480×360 matrix size, and an 8-bit gray scale with a time interval of 15 frames∕s of AFR from the original AVI cine clip which had an 800×600 matrix size with 15 or 30 frames∕s of the VFR. The matrix size of 480×360 was determined to exclude completely any patient information displayed on the original cine clip. The time interval in a series of images was standardized with 15 frames∕s of AFR by averaging of the series of input images if the original VFR was different from the AFR.

Because ultrasonography has very high time resolution, vascular flows represented in the MFI often had pulsation. In addition, electrical and cyclic signal noises due to harmonic imaging were seen frequently on the images. In order to reduce the signal noise and a cyclic change in pixel values between sequential images, and to enhance temporal changes due to vascularity inside and outside the FLL, we applied a smoothing filter and time-sequential running average techniques, respectively, to the series of input images. We used the smoothing-filter technique with a square kernel size of approximately 1.0 mm in which the matrix size was varied depending on the pixel size. The kernel size of 1.0 mm for the smoothing filter was determined based on the minimum size of the vessels displayed on ultrasound images. In the time-sequential running average technique, we produced a series of temporally averaged images in which a pixel value at each location was averaged with every five sequential frame images (the target frame and each of two frames before and after the target frame). The number of five frames (approximately 0.36 s) for the running average technique was determined because there were approximately three peaks of cyclic change per second in the original sequential frames due to pulsation. Figure 5 shows an example of temporal changes of differences in average pixel values in a FLL obtained with the series of the original input and temporally averaged images. Because the MFI applied a max-hold imaging technique, a rapid increase in the average pixel value at the first 1.0 s could be represented by small cyclic changes by use of temporally averaged images.

Figure 5.

Difference in average pixel values in sequential MFI images and replenishment times within a FLL (shown in the figure) obtained with the original and temporally averaged MFI images.

Image segmentation

In order to segment a kidney and vessel-like patterns, which often represent very high intensity in contrast-enhanced ultrasonography, from liver parenchyma, we used a multiple-gray-level thresholding³⁸ and a vessel-like pattern enhancement (VE) filter technique. These segmentations were determined by use of the last image in the series of MFI images with use of the contour of a FLL which was provided by an experienced physician.

A kidney and large vessel-like patterns due to the portal vein and inferior vena cava were initially segmented simply by use of a Gaussian filter and a multiple-gray-level thresholding technique.³⁸ Multiple-gray-level thresholding of the filtered image was employed for identifying initial candidates for a kidney or large vessel-like patterns. We automatically determined threshold pixel values in each step of the thresholding by use of areas under a histogram within a filtered image, the percentage of which ranged from 0.3% to 33% at high pixel values.³⁸ In this computerized scheme, initial candidates, which were derived by multiple-gray-level thresholding, were identified as a kidney or large vessel-like patterns if the effective diameter was 6.00 mm or greater.

We used the VE filter technique (Fig. 6), which can enhance and segment vessel-like patterns on the image. The output of the VE filter VE(i,j) at the location of pixel C(i,j) was defined as

$$\text{VE}\left(i,j\right)=\underset{\theta ,R}{\text{max}}\left\{-P_{\left(\theta ,R\right)}+2P_{\left(0,0\right)}-P_{\left(\theta +180,R\right)}\right\},$$

where P_(θ,R) is a pixel value at the location of (i+R sin θ,j+R cos θ), and the values of θ (deg) and R (mm) are {0, 45, 90, 135} and {2.0, 4.0, 8.0}, respectively. Thus, the output of the VE filter represented the maximum value among 12 s derivative values that varied with angles and filter sizes. By use of this filter technique, therefore, various sizes of vessel-like patterns could be enhanced in either a transverse or a longitudinal section.

Figure 6.

Explanation of the VE filtering technique for a calculation point C(i,j). P_(θ,R) is a pixel value at the location of (i+R sin θ,j+R cos θ). The VE filter output VE(i,j) was defined as the maximum value among 12 P_(θ,R) values obtained with four angles of θ and three filter sizes of R.

An ALP region was determined by use of histogram analysis of the Gaussian-filtered MFI images as well as segmented kidney, vessels, and the contour of an FLL provided by an experienced physician. In this computerized scheme, nonscanned regions in the MFI image were initially excluded by use of the threshold pixel value, which was determined at 2.0% of the maximum pixel value in the MFI image. The regions of segmented kidney, vessel-like patterns, and a FLL were excluded from all of the liver regions for determining ALP regions. Figure 7 shows an example of the original MFI, vessel, ALP, and skeletons of vessel-like pattern images. The purpose and the method of a skeleton of a vessel-like pattern image will be described in Sec. IV.

Figure 7.

Example of (a) original MFI image including one FLL (arrow) and a kidney, (b) vessel-like pattern enhanced image obtained with the VE filter technique, (c) segmented ALP regions obtained from the original MFI image, and (d) skeleton of vessel-like pattern enhanced image for estimating the average size of vessel-like patterns on the MFI image.

In addition, we divided a FLL region into central and peripheral regions by use of a fixed margin from the contour of the FLL, as shown in Fig. 8. The size of margin M was determined by a quarter of an effective diameter³⁹ of the FLL, which was defined with a diameter of a circle that had the same area as the FLL.

Figure 8.

Sample images of a FLL and illustration of a computerized scheme for dividing the FLL into central and peripheral regions. A fixed margin (M) was calculated from the effective diameter of a FLL and used for dividing the FLL. (a), (c) Original MFI image with a lesion contour which was drawn by the physician, and a contour of the central region segmented by the computer. (b), (d) Peripheral region of a FLL were divided from the central region by use of a rolling circle which has a radius M.

Estimation of time-intensity curves

The time-intensity curve with a theoretical model^{17, 34} for the FLL was estimated by use of average pixel values of the FLL on a series of temporally averaged MFI images. In this model, the time-intensity curve SI(t) was calculated automatically by use of a least-square method for the difference between the average pixel values at the replenishment time t and the theoretical model obtained with the following equation:¹⁷

$$\text{SI}\left(t\right)=H_{\text{max}}\left(1-e^{-\beta t}\right),$$

where H_max represents the peak pixel value in a region of interest, and β is a parameter corresponding to an exponential factor, which can be related to the flow rate in the first phase of the enhancement. By use of this model, temporal features of the peak pixel value H_max and the slope factor β of the FLL were determined. In addition to the temporal features, replenishment times at early and delayed phases of the MFI images were determined by use of the estimated time-intensity curve at 50% and 98% of the H_max value, respectively. In some specific FLLs such as hemangiomas, pixel values within the FLL were mostly saturated at the early phase and increased very slowly. In order to avoid including such unstable time data, we used 98% of the H_max value for the delayed phase instead of the use of 100%. On the other hand, if the estimated replenishment time for the delayed phase was longer than an actual acquisition time due to continuously increasing intensities, the time was determined to be the same as an actual acquisition time.

Extraction of morphologic and gray-level image features

In addition to the temporal features, we extracted morphologic and gray-level image features for the FLL by use of the original and temporally averaged MFI images at the early and delayed phases, and by use of the vessel-like pattern and ALP images.

We obtained three morphologic image features based on the contour of the FLL and a vessel image, such as (1) the effective diameter of a FLL, (2) the ratio of vessel-like pattern areas to the whole liver area within a MFI image, and (3) the average size of the vessel-like patterns. The average size of vessel-like patterns was determined by use of the vessel-like pattern and its skeleton images. Figure 7 shows an example of a skeleton image obtained from the vessel-like pattern image. In this method, we assumed that an approximate average size of a vessel-like pattern can be estimated by use of the ratio of the area of the skeleton image and the area of the vessel-like pattern image.

We also extracted three gray-level image features based on the MFI image at the early and the delayed phases, such as (1) the average pixel value without and with the segmented vessel-like patterns, (2) the standard deviation of pixel values without and with the segmented vessel-like patterns, and (3) two ratios of average pixel values for those in a FLL to an ALP, and for those in the central to the peripheral region of the FLL. The average pixel values were obtained from the whole and central∕peripheral regions of a FLL, whereas the others were obtained only from the whole region of a FLL.

Moreover, in order to obtain additional image features related to hypoechoic regions, we segmented hypoechoic regions within a region of the FLL. In this computerized scheme, the Gaussian-filter technique was applied to gray-scale-inverted MFI images at the early and the delayed phases. Because the replenishment of the contrast agent (microbubble) was imaged on the MFI image, vessels were imaged partially at the early phase and then were imaged fully at the delayed phase. The minimum size of the hepatic artery which could be imaged on MFI was approximately 3.0 mm. In order to remove noise components with enhancement of the hepatic arteries in two phases of the MFI images, therefore, we determined the kernel sizes for the Gaussian-filter technique with 2.0 and 3.0 mm for the MFI images obtained at the early and the delayed phases, respectively. The pixel value of the thresholding hypoechoic region on the filtered image was determined empirically as 192, and used for both filter sizes and all 103 FLLs. In other words, the regions with 25% or less than 25% of peak pixel values were considered to be a hypoechoic region. Figure 9 shows an example of a segmented hypoechoic region at the early and the delayed phases and the MFI image at the delayed phase. By use of the information about the segmented hypoechoic regions, we extracted additional image features, such as (1) the number of hypoechoic regions within a FLL, (2) the ratio of the area of hypoechoic regions to the whole area of the FLL, (3) the difference in the average pixel values between the early and the delayed phases, and (4) the average change in pixel values per second in delayed-enhancement regions. We defined a delayed-enhancement region when the region was included in the segmented hypoechoic region at the early phase, but not in the delayed phase, as shown in Fig. 9.

Figure 9.

Example of the original MFI image at the delayed phase and its segmented images for hypoechoic regions at the early and delayed phases. The difference in the regions between two images at two phases was defined as delayed-enhancement region.

Table 1 shows the total of 43 temporal and image features extracted initially in this computerized scheme.

Table 1.

List of 43 temporal and image features obtained from the original and temporally averaged images with whole, central, and peripheral regions. Features were extracted from a contour provided by a physician (C), a series of images (S), at the early-phase image (E), and at the delayed image (D). Bold type and a subscript number indicate the selected 19 features and ANN numbers. [C, S, E, and D represent the image(s) for feature extraction, such as a contour image, a series of images, an image at the early and the delayed phase, respectively. Bold type and following subscript number indicate a feature selected and its series number in six ANNs.]

	Original images			Temporally averaged images
		Whole	Central	Peripheral	Whole	Central	Peripheral
Temporal features
Replenishment time (s)				D3,5,6
Peak pixel value				S
Slope factor (β)				S
Morphologic features
Effective diameter of FLL	C1,4,5
Average size of vessel-like patterns				D2	D	D2
Area ratio of vessel-like patterns				D2	D2	D
Gray-level features
Average pixel value with vessel-like patterns	E, D	E, D3	E, D	E, D	E6, D	E6, D
Average pixel value without vessel-like   patterns				D3
Standard deviation of pixel value   with vessel-like patterns				E, D4
Standard deviation of pixel value   without vessel-like patterns				D5
Average pixel value ratio (FFL∕AHP)	E, D			E6, D
Average pixel value ratio (central∕peripheral)				E, D6
Features for hypoechoic region
Average pixel value				E, D
No. of hypoechoic regions				E, D
Area ratio of hypoechoic region				E, D	D1,4
Difference in pixel value (delay-early)				E∕D
Change in pixel value (delay-early)∕s				E∕D4

Feature selection and application of a cascade of ANNs

In order to classify five types of FLLs, we employed a cascade of six ANNs. Figure 10 illustrates a cascade of six ANNs used in this CAD scheme.

Figure 10.

Illustration of the cascade of six ANNs used in this study. Six decisions in which alternative choices for specific groups of FLLs were determined by single ALL, leading a final diagnostic decision for five liver diseases.

The number of ANNs of six was determined with the number of types of FLLs of seven (i.e., three types of HCCs, hyper- and hypovascularity metastasis, and hyper- and hypovascularity hemangioma). The order of the six decisions (D1–D6) at each ANN was determined based on the diagnostic difficulties, which were described in Sec. 2C.^{17, 18, 19, 23, 28} The six decisions used in this study are shown as follows:

• D1: Does this lesion have hypoechoic regions (yes) or not (no)?

• D2: Is this lesion a hypovascularity hemangioma (yes) or a hypovascularity metastasis (no)?

• D3: Is this lesion a hypervascularity hemangioma (yes) or other (no)?

• D4: Is this lesion a poorly differentiated HCC (yes) or other (no)?

• D5: Is this lesion a well differentiated HCC (yes) or other (no)?

• D6: Is this lesion a moderately differentiated HCC (yes) or a hypervascularity metastasis (no)?

All decisions were determined by ANNs in terms of a two-alternative choice. In all of the six ANNs, we employed one hidden layer, one half of the number of input units as the number of hidden units, one output unit, 0.05 for the learning rate, and 0.30 for the slope of the sigmoid function. These parameters were determined based on our experimental knowledge gained in previous research.³⁸ The seven combinations of the number of input units and iterations (2 and 200, 3 and 200, 3 and 300, 4 and 300, 4 and 400, 5 and 300, and 5 and 400) were tested for determining the appropriate parameters for each ANN. The maximum number of input unit was determined by five because the limited number of cases used in this study. The numbers of iterations corresponding to the number of input units were selected based on our experimental knowledge.³⁸

In order to select appropriate combinations of temporal and image features for each of the six ANNs, we used a stepwise method.⁴⁰ Although a conventional stepwise regression applies a sequence of F tests for determining the necessity and sufficiency of temporally selected features by use of their p values, we employed an ANN and the diagnostic accuracy estimated by the ANN instead of the F test and its p value. In this stepwise procedure, the feature selection was started by selection of the best candidate pair of feature combinations which had the maximum diagnostic accuracy in the training ANN with a selected number of iterations, as described above. In the forwarding selection of the stepwise after the best candidate pair of feature combinations had been selected, the remaining features were tried out one by one and considered as the best candidate combination if the diagnostic accuracy of this combination was greater than that with the last best candidate combination. In the backward selection, which involves starting with all features in the best candidate combination and testing them one by one with an ANN and estimating the diagnostic accuracy, deleting any if the diagnostic accuracy for the combination without a tested feature was not changed from that of the best candidate combination. Then, the forward and backward selections were repeated until the number of selected features reaches the preset maximum number of features, and the maximum diagnostic accuracy was not changed.

Finally, we selected the number of features (input units) and iterations, such as 2 and 200 for D1, 4 and 300 for D2, 3 and 300 for D3, 4 and 300 for D4, 4 and 400 for D5, and 5 and 400 for D6, and a total of 16 image features for the six ANNs. Table 1 also shows selected features of each ANN.

The six ANNs were trained and tested with a leave-one-lesion-out method independently by use of selected features and the number of iterations. In the leave-one-lesion-out test method used in this study, because the number of lesions subjected to training of each of the six ANNs were sometimes small and imbalanced, such as 81versus 22 for D1, 11 versus 11 for D2, 76 versus 5 for D3, 67 versus 9 for D4, 43 versus 24 for D5, and 28 versus 15 for D6, the numbers of positive and negative lesions in each of the six ANNs was matched to one another by duplicating of input data sets in order to reduce a bias due to the imbalance of the numbers of positive and negative lesions. The target value during the ANN for positive lesions (i.e., the specific group with answer “yes” to each decision) was 0.95, and 0.05 for negative (i.e., the specific group with answer “no” to each decision) lesions.

In order to use ANNs for making a two-alternative choice (i.e., yes or no) in each decision in the cascade of six ANNs, we used a fixed threshold value of 0.50 for distinguishing decisions made by the ANN output (i.e., 0.50 or more is yes, all others are no). The correct classification of the CAD scheme for three types of HCCs was determined when the final branch of a cascade of six ANNs which was led by a number of decisions agreed with its “gold standard.” Metastasis and hemangioma cases were considered as correctly classified by the CAD when its final branch was correctly selected from either FLLs without or those with hyperechoic regions. The classification accuracies for each type of FLL and also for all 103 FLLs were determined with percentages (%) of correctly classified cases among a total number of cases. Please note that we did not monitor the classification accuracies during training and testing of the six ANNs in the cascade.

RESULTS

The computer performance in terms of the number of FLLs correctly classified and the classification accuracies (%) in each of the six ANNs were 102 of 103 (99.3%) for D1, 21 of 22 (95.5%) for D2, 79 of 81 (97.5%) for D3, 69 of 76 (90.8%) for D4, 56 of 67 (83.6%) for D5, and 37 of 43 (86.1%) for D6.

Table 2 shows the performance of the computerized scheme for the classification of five types of FLLs (w-HCCs, m-HCCs, p-HCCs, metastases, and hemangiomas). The classification accuracies for the 103 FLLs were 88.5% for metastasis, 93.8% for hemangioma, 79.2% for w-HCC, 50.0% for m-HCC, and 77.8% for p-HCC. When the classification was done for three types of FLLs (HCCs, metastasis, and hemangioma), the classification accuracies for all HCCs was 86.9%. The average classification accuracies for three and five types of FLLs were 88.3% and 75.7%, respectively. Figure 11 shows examples of MFI images for each of the five diseases which were classified correctly by the CAD.

Table 2.

Number of FLLs and classification accuracies obtained by CAD for well differentiated HCCs, moderately differentiated HCCs, and poorly differentiated HCCs, metastases, and hemangiomas.

Type of disease			Classification with CAD
			HCC						Metastasis		Hemangioma
		No. of lesions	Well		Moderate		Poor
HCC	Well	24	19	(79.2%)	1	(4.2%)	2	(8.3%)	2	(8.3%)	0	(0.0%)
	Moderate	28	5	(17.9%)	14	(50.0%)	4	(14.3%)	3	(10.7%)	2	(7.1%)
	Poor	9	1	(11.1%)	0	(0.0%)	7	(77.8%)	1	(11.1%)	0	(0.0%)
Metastasis		26	2	(7.7%)	1	(3.8%)	0	(0.0%)	23	(88.5%)	0	(0.0%)
Hemangioma		16	0	(0.0%)	0	(0.0%)	0	(0.0%)	1	(6.3%)	15	(93.8%)

Figure 11.

Examples of correctly classified cases. (a) w-HCC showed relatively uniform and isoechoic replenishment patterns, and the size was relatively small (15.9 mm, pixel size: 0.29 mm). (b) m-HCC showed a relatively isoechoic replenishment pattern, but the size was large (23.4 mm, pixel size: 0.17 mm), and the replenishment pattern within a lesion was not uniform. (c) p-HCC had a relatively large size (20.1 mm, pixel size: 0.34 mm) and showed branched tumor vessels within a lesion. (d) Metastasis (hypovascularity type) had a large size (65.6 mm, pixel size: 0.29 mm) and showed a hypoechoic region at the center of the lesion at the delayed phase. And (e) hemangioma (hypovascularity type) had a relatively large size (56.6 mm, pixel size: 0.26 mm) and was replenished very slowly with centripetal progression.

As shown in Table 2, 12 FLLs, which were incorrectly classified into three types of liver diseases by the CAD, consisted of eight HCCs, three metastases, and one hemangioma. The CAD incorrectly classified eight HCCs as six metastases and two hemangiomas, three metastases as three HCCs, and one hemangioma as one metastasis. Therefore, the classification accuracies for malignant (HCCs and metastases) and for benign (hemangiomas) lesions were 97.7% and 93.8%, respectively.

DISCUSSION

In this CAD scheme, we employed a cascade of six independent ANNs. Because each ANN was designed specially for each decision by selection of appropriate image features, the majority of FLLs which had any of the typical patterns were classified correctly by the computer. In order to represent vascular patterns and temporal characteristics for all types of FLLs as close as possible to physicians’ diagnostic knowledge, we initially extracted 43 temporal and image features. However, only 16 features, including one temporal feature, five morphologic features, eight gray-level features, and two features for hypoechoic regions were selected for all ANNs (note that the maximum number of selected features for a single ANN was five). For example, in the first ANN for distinguishing FLLs with from those without hypoechoic regions, an image feature of the area ratio of the hypoechoic regions to that within a FLL was dominant among the selected two features. For example, 78 of 81 FLLs (61 HCCs and 15 hypervascularity metastases, and five hypervascularity hemangiomas) had no obvious hypoechoic regions at their delayed phases. For the second ANN, in a comparison between the hypovascularity hemangiomas and the hypovascularity metastases, tumor vessels in the central region of the hypovascularity hemangiomas were likely not to be enhanced within an acquisition time (range from 4.1 to 23.1 s) of the MFI. Thus, morphologic features related to vessel-like patterns in the central and peripheral regions were effective for distinguishing them. For distinguishing hypervascularity hemangiomas in the third ANN, a temporal feature of the estimated replenishment time at the delayed phase was effective because the estimated replenishment time for hemangioma was likely to be long compared with that for the other types of liver diseases. The p-HCCs were likely to be large and to indicate a heterogeneous enhancement pattern due to thick tumor vessels compared with w-HCCs, m-HCCs, and hypervascularity metastases. Therefore, the effective diameter and the standard deviation of pixel values in a FLL with vessel-like pattern regions were effective in the fourth ANN. For distinguishing w-HCCs from m-HCCs and hypervascularity metastases, the standard deviation of pixel values in a FLL without vessel-like pattern regions and the estimated replenishment time at the delayed phase was effective because w-HCCs were likely to be enhanced uniformly and slowly compared with those with m-HCCs and with hypervascularity metastases. It is well known that the distinction between m-HCCs and hypervascularity metastases was very difficult for physicians.¹¹ However, as shown in Fig. 2, some m-HCCs were likely to be enhanced with centrifugal or centripetal patterns; therefore, we selected the ratio of the average pixel values in the central region of a FLL to that in the peripheral region. Figure 12 shows one example of a metastasis case which was classified incorrectly as w-HCC. As shown in Fig. 12, it is very difficult to distinguish hypervascularity metastases from HCCs because they show identical vascularity patterns in the arterial phase.²⁴

Figure 12.

Example of the CAD output for a FLL (hypervascularity metastasis) which was incorrectly classified as m-HCC, because the size of this lesion was relatively large (25.1 mm, pixel size: 0.24 mm), and the echogenicity of this metastasis was isoechoic to those in ALP regions.

The performance of our CAD scheme for the classification of FLLs could be considered as comparable to those reported by Wilson and Burns.²³ Although their performance was not the results of the CAD, their algorithm used the subjective assessment of physicians for information on portal venous enhancement in the distinction between benign and malignant FLLs and indicated a very high classification accuracy (i.e., 92% for benign and 93% for malignant FLLs),²² which was evaluated by use of a resubstitution method.

In this study, we used MFI images which were selected from either the arterial or the portal venous phase of contrast-enhancement ultrasonography because the appropriate enhancement phase for distinguishing FLLs depended on the type of disease as well as the physicians’ judgment. In addition, locations and contours of FLLs were determined by only one physician in this study. Although the margin of each FLL was drawn concisely, as shown in Figs. 7810, and we did not use the shape of FLLs (such as circularity and irregularity) for the image feature analysis, there would be some variations in the performance of the CAD scheme due to the physician dependence. Furthermore, the quality of the MFI images depended on the patients’ condition. Therefore, as is well-known, a potential limitation for all ultrasonographic studies is that the image acquisition of the MFI is operator dependent.

Replenishment vascularity patterns in the MFI with second-generation perfluorocarbon agents were slightly different from those in the arterial and portal vein phases in pulse-inversion imaging with the second-generation air-based contrast agent in terms of continuous imaging of inflow high signals. However, most of the vascularity patterns for FLLs were common to both imaging methods, so that we could take into account some specific vascularity patterns in our computerized scheme.

In this study, the gold standard for evaluating the computerized scheme was determined by pathology. However, because we used the MFI images obtained from only one section, mixtures of two or three histologic differentiation types in a FLL were not taken into account in our evaluation. In addition, because specimens for histopathologic diagnosis of the degree of differentiation of HCCs were obtained by use of a fine-needle biopsy, the specimen allowed us to evaluate only a small part of the HCC lesion. Therefore, the computer performance for the classification between three differentiated types of HCCs was potentially underestimated.

There are several limitations to this CAD study. On the one hand, the number of cases used in this study was relatively small, and only five types of liver diseases were included in the database. For example, we did not include FNH because the number of FNH cases was very small. However, we believe that the computerized scheme with MFI can be modified for classifying six types of liver diseases, including FNH cases when the number of FNH cases is increased because the image characteristics of FNH in contrast-enhanced ultrasonography also have been studied thoroughly.^{10, 17, 23}

Other limitations were that feature selection and parameter settings were determined empirically and by use of a stepwise method for each ANN, even though six ANNs in the cascade were trained and tested by a leave-one-lesion-out method by use of selected features, and the final result obtained from the cascade of six ANNs was not monitored during the training and testing of six ANNs. This feature selection method depended on the database used, and thus, there could be some bias in the evaluation of results.⁴¹ However, a number of features were reasonably selected in each ANN in terms of the agreement with physicians’ knowledge, and thus, physicians’ confidence in the use of the CAD scheme would be improved. In addition, a number of parameters used in this CAD scheme were determined empirically or based on our experimental knowledge. For justifying the determination of these parameters, further systematic study with a large-scale image database would be necessary in the future.

CONCLUSION

We developed a computerized scheme for the classification of focal liver lesions by use of MFI of contrast-enhanced ultrasonography. The performance of this CAD scheme has the potential to improve the diagnostic accuracy of the histologic characteristics of HCCs and other liver diseases.