Abstract
Background & Aims
Enhanced computed tomography (CT) is the primary method for focal liver lesion diagnosis. We aimed to use automated machine learning (AutoML) algorithms to differentiate between benign and malignant focal liver lesions on the basis of radiomics from unenhanced CT images.
Methods
We enrolled 260 patients from 2 medical centers who underwent CT examinations between January 2017 and March 2023. This included 60 cases of hepatic malignancies, 93 cases of hepatic hemangiomas, 48 cases of hepatic abscesses, and 84 cases of hepatic cysts. The Pyradiomics method was used to extract radiomics features from unenhanced CT images. By using the mljar-supervised (MLJAR) AutoML framework, clinical, radiomics, and fusion models combining clinical and radiomics features were established.
Results
In the training and validation sets, the area under the curve (AUC) values for the clinical, radiomics, and fusion models exceeded 0.900. In the external testing set, the respective AUC values for the clinical, radiomics, and fusion models were as follows: 0.88, 1.00, and 1.00 for hepatic cysts; 0.81, 0.90, and 0.97 for hepatic hemangiomas; 0.89, 0.98, and 0.92 for hepatic abscesses; and 0.23, 0.80, and 0.93 for hepatic malignancies. The diagnostic accuracy rates for hepatic cysts, hemangiomas, malignancies, and abscesses by radiologists in the external testing cohort were 0.96, 0.60, 0.79, and 0.66, respectively.
Conclusion
The fusion model based on noninvasive radiomics and clinical features of unenhanced CT images has high clinical value for distinguishing focal hepatic lesions.
Keywords: Liver, Machine learning, Radiomics, CT scan, Carcinoma
Lay summary
The preoperative diagnosis of focal liver lesions is essential for choosing appropriate treatment. Thus, we aimed to use the MLJAR AutoML framework to differentiate benign and malignant focal liver lesions on the basis of radiomics from unenhanced CT images.
Introduction
Liver lesions are frequently encountered by radiologists. They have complex pathologies ranging from isolated benign lesions to primary liver cancer and metastases [1]. Focal hepatic lesions are often discovered incidentally via cross-sectional imaging or abdominal ultrasound. In the general population, most of those incidental findings are benign, which are usually asymptomatic and sporadic [2, 3]. Benign liver lesions include focal nodular hyperplasia (FNH), hepatocellular adenoma, hepatic cysts (HC), hepatic hemangiomas (HH), hepatic abscesses (HA), and pseudotumors [4]. Malignant tumors include hepatocellular carcinoma (HCC), cholangiocarcinoma, and hepatic metastases [5].
Accurately identifying these lesions is crucial for diagnosis and prognosis [6]. The preoperative imaging of focal liver lesions is essential for choosing appropriate treatment. However, the preoperative imaging diagnosis of benign and malignant focal liver lesions is difficult and sometimes impossible, particularly with atypical or multiple lesions [7]. We designed this model to classify the lesions more quickly and accurately in unenhanced CT. Owing to the inherent characteristics of contrast uptake and washout during each phase, liver lesions can be detected and characterized using multiphase contrast-enhanced CT. Enhanced CT can provide more information than unenhanced CT for focal liver lesions, but the iodine-containing contrast agents required for enhanced CT are sensitized, nephrotoxic, and carcinogenic. However, anaphylactoid reactions are known complications of intravenous iodinated contrast media and range from mild symptoms such as hives and itching to more severe reactions such as cardiac arrest and death [8]. Severe allergic reactions rarely occur, but when they do occur they are very critical and develop rapidly, requiring emergency rescue and hospitalization for observation. For people with chronic kidney disease, the elderly and chronic heart failure, the incidence of contrast nephropathy can reach 20%, and the incidence of renal insufficiency is 50%. Therefore, using unenhanced CT to conduct this study will have more clinical significance [9].
Medical images provide information reflecting underlying pathophysiology, which can be revealed through quantitative image analysis [10]. Various qualitative and quantitative imaging techniques have been developed to overcome the shortcomings of liver biopsy. Among these techniques, ultrasound elastography is the most widely used in clinical practice because of its convenience. Compared with other modalities, CT is relatively easy to perform, is standardized for different scanners, is fast, and has high temporal and spatial resolutions [11]. Unenhanced CT is performed without exposing patients to potentially allergenic or toxic contrast agents [8]. Magnetic resonance imaging (MRI) is often used to characterize indeterminate liver lesions because of high image contrast and the absence of ionizing radiation. However, MRI requires more time and is susceptible to respiratory motion artifacts [12].
On the basis of these shortcomings, we attempted to apply radiomics to enhance diagnostic efficacy. Radiomics, which is the conversion of digital medical images into high-dimensional data that can be extracted and analyzed, has rapidly advanced in medicine in recent years [13–15] and has been used to identify quantitative imaging features predictive of tumors; treatment responses; and overall outcomes in various malignancies, including breast, pancreatic, and lung cancer [16–18]. Many studies have been published on the application of radiomics in focal hepatic lesions; however, only a few have focused on diagnosing these lesions using unenhanced CT images, and most were single-center studies [19].
The aim of this study was to conduct a multicenter investigation using radiomics-based methods on unenhanced CT scans to distinguish hepatic malignancy (HM), HH, HC, and HA. The noninvasive radiomics features can be used to improve diagnosis and predict prognosis [20, 21].
Methods
This retrospective study was approved by the Institutional Review Board (no. 20240045) of our hospital and was conducted in accordance with the Declaration of Helsinki. Patient consent was waived owing to the retrospective nature of this study.
Patients
We included 423 patients with HM, HH, HC, or HA who underwent CT at our hospital (Medical Center A) between January 2017 and March 2023. Additionally, we included 40 patients with similar conditions who underwent CT examination at another hospital (Medical Center B) between January 2022 and March 2023.
The inclusion criteria were as follows: (1) upper abdominal unenhanced and enhanced CT performed simultaneously or no more than 30 days apart, (2) liver-occupying lesions ≤ 10, and (3) good image quality meeting labeling requirements of radiologists.
The exclusion criteria were as follows: (1) lesions < 1.0 cm, (2) patients with iodide deposits from interventional therapy, (3) absence of standard abdominal unenhanced CT, and (4) hepatic malignancy without pathological confirmation. The study flowchart, from data collection to evaluation, is presented in Fig. 1.
Fig. 1.
Flowchart of the study from data collection to evaluation. Pn, Number of patients; Ln, Number of lesions; HC, Hepatic cyst; HH, Hepatic hemangioma; HA, Hepatic abscess; HM, Hepatic malignancy
Hepatic lesion confirmation
Hepatic lesions were confirmed histopathologically through surgery or percutaneous needle biopsy [22]. If pathological results were unavailable, typical MRI or CT findings were used to characterize the lesions [23]. HH, HC, and HA were confirmed by referencing radiologic reports by experienced radiologists and adhering to the following criteria [7, 24]: HH—CT shows a hypodense, well-defined lesion with internal density similar to vessels and peripheral nodule enhancement in the arterial phase, and cardiac filling enhancement in the venous phase [15]; HC—CT confirms water density (attenuation < 20 HU) with clear edges and no enhancement after contrast administration [20]; HA—Imaging findings of HA are closely related to the pathological stage. Unenhanced CT shows heterogeneous, low-density lesions with unclear boundaries during early pyogenic stages. In the suppuration stage, the density is lower than surrounding normal liver parenchyma, with a thin wall and clear boundary [25, 26].
Several types of focal hepatic lesions were classified into benign and malignant groups. The benign group includes HH, HC, and HA, whereas HM belongs to the malignant group [27].
CT image acquisition
CT scans were performed using post-64-detector row CT scanners from Siemens (Somatom Definition Flash, Somatom Force, or Somatom Drive, Forchheim, Germany) and GE (Revolution CT, Discovery CT750 HD, or 64-slice LightSpeed VCT, GE Medical Systems, Milwaukee, WI). Imaging data were reconstructed using a 1 mm medium sharp algorithm. The other scanning parameters included rotation time, 0.5 s; pitch, 1.2–1.375; matrix, 512 × 512; standard resolution algorithms; and tube voltage, 80–100 kV (Somatom Definition Flash or Somatom Force or Somatom Drive) and 120 kVp (Revolution CT, Discovery CT750 HD or 64-slice LightSpeed VCT, GE Healthcare). The tube current was automatically adjusted in the noise index mode. Enhanced scanning was performed by injecting a nonionic contrast agent (iodine content, 320 g/L) into the cubital vein at 2.5–4.0 mL/s; the total calculated dose was 1.5 mL/kg. After the contrast agent was injected, the arterial, venous, and delayed phases were scanned between 25 and 30 s, after 60 s, and after 180 s, respectively.
Clinical information acquisition
Patient demographic and clinical data, including age, sex, and pathological results, were recorded from picture archiving and communication systems; tumor location, mean CT value, size, and morphology were assessed from unenhanced CT images. Tumor location was evaluated on the basis of liver capsule protrusion, and tumor size was measured from the largest boundary of the lesion’s region of interest (ROI) in clinical settings. Tumor morphology was evaluated for regularity (round shape) and boundary clarity. All measurements were completed simultaneously by the labeling radiologists.
Radiologists image evaluation
One board-certified abdominal radiologist and one second-year radiology resident (S.Y.L., seven years of experience in digestive system radiology; L.D.C., two years of experience in digestive system radiology), both blinded to hepatic lesion outcomes, independently reviewed the axial unenhanced CT images of each lesion in the external testing cohort. The radiologists scored the probability of each lesion being a cyst, hemangioma, malignancy, or abscess.
Tumor segmentation
Two radiologists (Y.N., two years of experience in radiology; M.Z.X., six years of experience in radiology) used open-source software (3D Slicer, version 4.13.0; National Institutes of Health; https://www.slicer.org; accessed on August 7, 2021) to manually delineate the volume of interest (VOI) for focal hepatic lesions. At least one type of focal liver lesion was selected for each patient, with the largest lesion of each type chosen for segmentation. The radiologists manually delineated the ROI along the edge of the lesion layer-by-layer on unenhanced CT images, and the VOIs were automatically generated by a computer. The results were reviewed by a senior radiologist (L.M., with > 20 years of experience in digestive system radiology). During segmentation, the corresponding enhanced CT images were used to determine tumor boundaries.
Radiomic feature extraction
Radiomic features were extracted from 3D ROIs using Pyradiomics (version 3.0.1) to comply with the standards of the image biomarker standardisation initiative. Feature selection is a key step in the AutoML, which aims to identify and select those features from the raw data set that have a significant impact on the performance of the model. Through feature selection, redundant features can be reduced, model complexity can be reduced, and model training speed and generalization ability can be improved. MLJAR AutoML is an automated machine learning tool that automates feature selection tasks to simplify machine learning workflows. The radiomics features selected by AutoML, including shape features, First-order feature, Second-order features and Higher-order features. Parameters were set as follows: Spatial Resampling, 1 mm × 1 mm × 1 mm; Intensity Rescaling, 500; and Intensity Discretization, bin width of 25.
Consistency of segmentation and radiomics features
The intraclass correlation coefficient (ICC) evaluated the reliability of radiomics values between the two radiologists. The ICC measures and evaluates interobserver and test–retest reliability. Here, the ICC was calculated using a single measurement, absolute agreement, and a two-way random-effects model. Initially, VOI segmentations in the 227 patients, including 252 lesions, were performed by 2 radiologists. For reliability evaluation, 30 random CT images were selected and analyzed by another radiologist (J.L., with > 10 years of experience in digestive system radiology).
Automated machine learning (AutoML) model design
The mljar-supervised (MLJAR) platform is an AutoML Python package that works with tabular data. It was designed to save time for data scientists. It abstracts a common way to preprocess the data, construct machine learning models, and perform hyperparameter tuning to find the best model [28, 29].
The entire dataset from Medical Center A was randomly split into a training set (n = 176; HC = 54, hepatic malignancy = 62, HA = 28, HH = 32) and an independent validation set (n = 76; HC = 23, hepatic malignancy = 29, HA = 12, HH = 12). Additionally, 33 patients from Medical Center B formed the external testing set (n = 33; HC = 23, hepatic malignancy = 29, HA = 12, HH = 12). The radiomics workflow based on the automated learning algorithm is shown in Fig. 2. Three predictive models were developed—a radiomics model trained on radiomics features, a clinical model using only clinical features, and a fusion model incorporating both features.
Fig. 2.
Flowchart of the study. (a) Radiologists performed tumor segmentation on unenhanced CT. (b) Clinical information acquisition and radiomics feature extraction. (c) Automatic machine learning algorithms used to establish clinical, radiomics, and fusion models and complete predictive evaluation
An AutoML algorithm was designed to operate without human intervention to build the prediction model. It automatically screens features for participating in training, selects the model, and adjusts model parameters dynamically, thus significantly reducing the time and technical cost of the application. During data preprocessing, all features were normalized to zero mean and unit variance, with missing value imputation and conversion to categories handled automatically. The golden feature algorithm was used in the feature selection process. The MLJAR AutoML framework uses numeric features in the golden feature search. From each pair of original features, a new feature was created using the mathematical operators +, -, and /. A decision-tree algorithm was used to assess the predictive power of the newly created features, including only the top new features in the training dataset. The golden feature method maximizes the use of lesion information. In this study, MLJAR AutoML adopted Bayesian Optimization when automatically adjusting parameters: Bayesian optimization is a probabilistic model-based optimization method that uses past evaluation results to guide subsequent parameter selection, approximates the objective function by building an alternative function (probabilistic model), and optimizes the parameters through continuous iteration. The parameter adjustment process of MLJAR AutoML in this study includes the following steps: define the parameter space, select optimization algorithm, combination of evaluation parameters, iterative optimization, model training and verification.
The importance was computed using permutation, with dependence and decision plots for every algorithm available for analysis. Models were trained using various algorithms, including Nearest Neighbors, Linear, Random Forest, Extra Trees, LightGBM, Xgboost, and CatBoost. Hyperparameter optimization was conducted using a random search over defined values, the Optuna framework, and hill-climbing to fine-tune the final models.
Statistical analysis
Statistical analyses were performed using R (version 4.2.2) (https://www.r-project.org), Python (version 3.9.7), and SPSS (version 26) with significance set at P < 0.05. The performance of the prediction models was assessed using several indices with 10-fold cross-validation of the training and validation sets. Receiver operating characteristic curves were also used to assess the overall performance of prediction models, and the area under the curve (AUC) was calculated.
Results
Patients
This study retrospectively analyzed 252 focal liver lesions in 227 patients at Medical Center A and 33 lesions in 33 patients at Medical Center B. The 252 cases from center A included 79, 81, 40, and 52 cases of HC, HH, HA, and HM, respectively. At Medical Center B, there were 8, 12, 5, and 8 cases of HC, HH, HA, and HM, respectively. Patient numbers are presented in Table 1.
Table 1.
Patient cohort information
| Tumor types | Medical center A (Pn = 227,Ln = 252) |
Medical center B (Pn = 33, Ln = 33) |
|---|---|---|
| Hepatic cyst (HC) | 79 | 8 |
| Hepatic hemangioma (HH) | 81 | 12 |
| Hepatic abscess (HA) | 40 | 5 |
| Hepatic malignancy (HM) | 52 (20.6%) | 8 (20.6%) |
|
Benign hepatic tumors (HB) (HC + HH + HA) |
200 (79.4%) | 25 (20.6%) |
Pn, Number of patients; Ln, Number of lesions. The statistics in the table are the number of lesions
Clinical and CT image characteristics analyses
The clinical and radiological characteristics of patients are summarized in Table 2. There were no significant differences in the clinical characteristics (including sex and age) of patients with HC, HH, HA, or HM between medical centers A and B (p > 0.05).
Table 2.
Patient characteristics for each dataset
| Medical Center A (Ln = 252) | HC | HH | HA | HM |
|---|---|---|---|---|
| Clinical Characteristics | ||||
Age (Mean SD) |
69.33  11.74 |
59.93 13.34 |
64.13 16.78 |
69.90 11.55 |
| Sex: Male (n) | 41 (51.9%) | 38 (46.9%) | 21 (52.5%) | 33 (63.5%) |
| Female (n) | 38 (48.1%) | 43 (53.1%) | 19 (47.5%) | 19 (36.5%) |
| Radiologic Characteristics | ||||
| Multiple lesions (n)* | 60 (75.9%) | 30 (37.0%) | 8 (20.0%) | 28 (53.8%) |
| Morphological rule (n)* | 78 (98.7%) | 19 (23.2%) | 19 (47.5%) | 27 (51.9%) |
| Sharpness of border (n)* | 75 (94.9%) | 7 (8.6%) | 20 (50%) | 9 (17.3%) |
| Protrusion (n)* | 13 (16.5%) | 14 (17.3%) | 7 (17.5%) | 35 (67.3%) |
| Maximum diameter of lesion (mm)* | 19.34 27.53 |
19.78 27.14 |
56.08 20.66 |
54.62 27.85 |
| Mean CT value of lesion (HU)* | 10.86 16.28 |
39.86 14.40 |
27.33 6.01 |
37.27 16.13 |
| Medical Center B (Ln = 33) | HC | HH | HA | HM |
| Clinical Characteristics | ||||
Age (Mean SD) |
69.00 9.94 |
55.17 17.83 |
57.00 18.76 |
59.13 12.47 |
| Sex: Male (n) | 1 (12.5%) | 4 (33.3%) | 3 (48.5%) | 8 (48.5%) |
| Female(n) | 7 (87.5%) | 8 (66.7%) | 2 (51.5%) | 0 (51.5%) |
| Radiologic Characteristics | ||||
| Multiple lesions (n)* | 0 (0.0%) | 1 (8.3%) | 0 (0.0%) | 0 (0.0%) |
| Morphological rule (n)* | 8 (100.0%) | 12 (100.0%) | 1 (20.0%) | 5 (62.5%) |
| Sharpness of border (n)* | 8 (100.0%) | 12 (100.0%) | 1 (20.0%) | 2 (25.0%) |
| Protrusion (n)* | 1 (27.4%) | 3 (25.0%) | 1 (20.0%) | 2 (25.0%) |
| Maximum diameter of lesion (mm)* | 15.13 5.13 |
27.75 19.46 |
49.40 22.40 |
52.38 23.49 |
| Mean CT value of lesion (HU)* | 11.25 5.07 |
36.33 6.28 |
29.60 14.13 |
42.25 6.98 |
HC, Hepatic cyst: HH, Hepatic hemangioma; HA, Hepatic abscess; HM, Hepatic malignancy; Ln, Number of lesions. The statistics in the table are the number of lesions.Marked with * means P value < 0.05 by chi-square test or t test
Regarding radiological characteristics, cases obtained from Medical Center A had multiple lesions, with 75% of HC presenting multiple lesions. On unenhanced CT, almost all HC cases showed regular morphology and clear boundaries, with only a few growing under and protruding from the liver capsule. Among the four types of focal liver lesions, HC had the lowest mean CT value on unenhanced CT, with an average value of approximately 11 HU.
HH were usually single, located under the capsule, and showed slightly high density on unenhanced CT, with average CT values of approximately 36–40 HU. HA and HM were larger in diameter and lacked regular morphology on unenhanced CT. The average CT value of HA was lower than that of HH and HM. HM exhibited infiltrative growth, were mostly located under the liver capsule, and had irregular morphology, unclear boundaries, and varying CT values.
Radiomic features extraction
A total of 1218 radiomic features were extracted, including 18 first-order statistics; 14 shape statistics; 68 texture features including gray-level co-occurrence matrix, gray-level size zone matrix, gray-level dependence matrix, and gray-level run length matrix; 430 statistical features derived from texture matrices in the Laplacian of Gaussian filtered domain (1.0–5.0 mm kernels); and 688 statistical features from texture matrices in wavelet filtered domains. Owing to the lengthy names of the radiomic features, they were replaced with Arabic numerals.
Feature selection and golden feature establishment
In the clinical model, the two features with the highest importance coefficients were mean CT value and maximum lesion diameter. Sex and age were less helpful in distinguishing the four focal liver lesions on unenhanced CT images. The MLJAR AutoML framework automatically identified several optimal clinical features for model performance, as illustrated in Fig. 3A.
Fig. 3.
Feature importance coefficient map. (a) Clinical characteristics and (b) radiological characteristics
In the radiomics model, the top five importance coefficients were the following:
Wavelet_LLL_firstorder_10Percentile,
Wavelet_LLL_gldm_DependenceEntropy,
Wavelet_LLH_glcm_JointAverage,
Wavelet_LLL_glcm_ClusterProminence,
Original_glcm_ClusterProminence.
Figure 3B shows the top 25 radiomic features screened by MLJAR AutoML.
Diagnostic performance of autoML models
In the clinical model, the AUC for all four intrahepatic space-occupying lesions exceeded 0.9. However, in the external testing set, the diagnostic ability for HM was poor (AUC = 0.23). The low accuracy of the clinical model due to the small sample size of the external testing set may be the main reason for the low AUC value. The other four indicators (precision, recall, F1-score, and support) are summarized in Table 3.
Table 3.
Diagnostic performance of automatic machine learning models based on clinical features
| Category | Tumor types | support | precision | recall | F1 | AUC |
|---|---|---|---|---|---|---|
|
Training (n = 176) |
Hepatic cyst (HC) | 54 | 0.96 | 1.00 | 0.98 | 1.00 |
| Hepatic hemangioma (HH) | 62 | 0.96 | 0.84 | 0.90 | 0.98 | |
| Hepatic abscess (HA) | 28 | 0.85 | 0.82 | 0.84 | 0.98 | |
| Hepatic malignancy (HM) | 32 | 0.77 | 0.94 | 0.85 | 0.97 | |
|
Verification (n = 76) |
Hepatic cyst (HC) | 23 | 1.00 | 1.00 | 1.00 | 1.00 |
| Hepatic hemangioma (HH) | 29 | 0.93 | 0.86 | 0.89 | 0.97 | |
| Hepatic abscess (HA) | 12 | 0.85 | 0.92 | 0.88 | 0.95 | |
| Hepatic malignancy (HM) | 12 | 0.77 | 0.83 | 0.80 | 0.96 | |
|
Testing (n = 33) |
Hepatic cyst (HC) | 8 | 0.75 | 0.67 | 0.71 | 0.88 |
| Hepatic hemangioma (HH) | 12 | 0.67 | 0.80 | 0.73 | 0.81 | |
| Hepatic abscess (HA) | 5 | 0.80 | 0.44 | 0.57 | 0.89 | |
| Hepatic malignancy (HM) | 8 | 0.00 | 0.00 | 0.00 | 0.23 |
Support: can be defined as the corresponding number of samples in each class of target values. F1-Score: refers to the harmonic average of accuracy and recall, which is used to balance the influence between the two
The diagnostic efficiency of the radiomics model was better than that of the simple clinical model. In the external testing set, the radiomics model had an AUC of 0.8 for HM. The other four indicators of the radiomics model (precision, recall, F1-score, and support) are summarized in Table 4. The fusion model combines clinical and radiomics features. In the training, validation, and external testing sets, the AUC performance of each lesion was greater than 0.9; the detailed model results are shown in Tables 4 and 5. Figures 4 and 5 display the outcomes of the three prediction models. The reason for the high accuracy of hepatic cysts compared with other types of focal liver lesions is mainly due to the large differences in imaging and histopathology between hepatic cysts and the other three types of lesions [30].
Table 4.
Diagnostic performance of automated machine learning models based on radiomics features
| Category | Tumor types | support | precision | recall | F1 | AUC |
|---|---|---|---|---|---|---|
|
Training (n = 176) |
Hepatic cyst (HC) | 54 | 0.98 | 0.96 | 0.97 | 1.00 |
| Hepatic hemangioma (HH) | 62 | 0.91 | 0.92 | 0.92 | 0.98 | |
| Hepatic abscess (HA) | 28 | 0.89 | 1.00 | 0.94 | 0.99 | |
| Hepatic malignancy (HM) | 32 | 0.95 | 0.88 | 0.91 | 0.98 | |
|
Verification (n = 76) |
Hepatic cyst (HC) | 23 | 0.96 | 1.00 | 0.98 | 1.00 |
| Hepatic hemangioma (HH) | 29 | 0.85 | 0.96 | 0.90 | 0.99 | |
| Hepatic abscess (HA) | 12 | 0.85 | 0.92 | 0.88 | 0.98 | |
| Hepatic malignancy (HM) | 12 | 1.00 | 0.72 | 0.84 | 0.97 | |
|
Testing (n = 33) |
Hepatic cyst (HC) | 8 | 1.00 | 1.00 | 1.00 | 1.00 |
| Hepatic hemangioma (HH) | 12 | 0.67 | 0.80 | 0.73 | 0.90 | |
| Hepatic abscess (HA) | 5 | 0.80 | 0.80 | 0.80 | 0.98 | |
| Hepatic malignancy (HM) | 8 | 0.62 | 0.50 | 0.56 | 0.80 |
Support: can be defined as the corresponding number of samples in each class of target values. F1-Score: refers to the harmonic average of accuracy and recall, which is used to balance the influence between the two
Table 5.
Diagnostic performance of automatic machine learning models based on fusion features
| Category | Tumor types | support | precision | recall | F1 | AUC |
|---|---|---|---|---|---|---|
|
Training (n = 176) |
Hepatic cyst (HC) | 54 | 1.00 | 0.97 | 0.98 | 1.00 |
| Hepatic hemangioma (HH) | 62 | 0.93 | 0.88 | 0.90 | 0.99 | |
| Hepatic abscess (HA) | 28 | 0.81 | 0.92 | 0.86 | 0.98 | |
| Hepatic malignancy (HM) | 32 | 0.82 | 0.86 | 0.84 | 0.98 | |
|
Verification (n = 76) |
Hepatic cyst (HC) | 23 | 1.00 | 1.00 | 1.00 | 1.00 |
| Hepatic hemangioma (HH) | 29 | 0.89 | 1.00 | 0.94 | 0.99 | |
| Hepatic abscess (HA) | 12 | 0.92 | 0.80 | 0.86 | 0.98 | |
| Hepatic malignancy (HM) | 12 | 0.92 | 0.80 | 0.86 | 0.98 | |
|
Testing (n = 33) |
Hepatic cyst (HC) | 8 | 1.00 | 1.00 | 1.00 | 1.00 |
| Hepatic hemangioma (HH) | 12 | 0.92 | 0.92 | 0.92 | 0.97 | |
| Hepatic abscess (HA) | 5 | 0.75 | 0.75 | 0.75 | 0.92 | |
| Hepatic malignancy (HM) | 8 | 0.80 | 0.80 | 0.80 | 0.93 |
Support: can be defined as the corresponding number of samples in each class of target values. F1-Score: refers to the harmonic average of accuracy and recall, which is used to balance the influence between the two
Fig. 4.
Diagnostic performance of automated machine learning models. (a) Clinical model, (b) Radiomics model, (c) Fusion model. The optimal model is indicated by the red box
Fig. 5.
Diagnostic performance of automated machine learning models
Performance of radiologists in the external testing cohort
Two radiologists evaluated diagnoses using external testing cohort. The correct identification of focal liver lesion was one point. If they could not make a clear diagnosis, the probability of the lesions they considered likely was recorded. Finally, all probabilities were compiled to obtain the accuracy rate of the radiologists’ diagnoses for the different focal liver lesions. The diagnostic accuracy rates for HC, HH, HM, and HA in the external testing cohort were 0.96, 0.60, 0.79, and 0.66, respectively. In the external testing cohort, the accuracy of fusion model in identifying lesions was 1.00, 0.97, 0.93 and 0.92, respectively, the radiomics model was 1.00, 0.90, 0.80 and 0.98, and the clinical model was 0.88, 0.81, 0.23 and 0.89, respectively. From this, we can see that the accuracy of our model is higher than that of radiologists, especially for hepatic hemangiomas and liver abscesses.
Discussion
Owing to the advancements in artificial intelligence (AI) techniques, radiomics has become increasingly prevalent in clinical settings. Our study presents a radiomics-based AutoML model to aid in the clinical differentiation of focal hepatic lesions on unenhanced CT images, specifically distinguishing between HM, HH, HC, and HA.
Enhanced CT has long been the preferred diagnostic method for focal liver lesions [31]. However, enhanced CT examination involves increased exposure to X-rays and adverse reactions to contrast media [13]. These reactions include nausea and vomiting, dizziness, headache, rash, severe blood pressure, convulsions, shock, and even death [26]. These risks are particularly concerning for patients with glomerular filtration rates below 30 mL/min/1.73 m2, with a significant trend at 30–44 mL/min/1.73 m2 [32]. Therefore, this study utilized unenhanced CT owing to its easy accessibility and lower radiation dose to avoid the shortcomings of contrast media while still providing important visual clues.
Radiomics-based AutoML techniques facilitate the extraction of high-throughput features from medical images by converting them into quantitative data [33]. These techniques reflect the morphological features of lesions visible to the naked eye and microscopic features. Hu et al. reported a computer-aided diagnosis-based method for distinguishing HCC from HH by using unenhanced CT [34] with a two-stage model, including a radiomics signature and radiomics index. In the validation group, the radiomics signature and index achieved excellent diagnostic performance with AUC values of 0.716 and 0.870, respectively [35].
Zhao et al. explored the feasibility of the computer-aided differential diagnosis of HCC, hepatic metastases, HH, HC, hepatic adenomas, and FNH by using the radiomic analysis of unenhanced CT. Support Vector Machine was used to establish the classifier. The overall accuracy of the training and test sets for differentiating the six histopathological types of lesions was 0.88 and 0.76, respectively [36]. Huang et al. used computer-aided diagnosis methods to distinguish liver malignant tumors and liver hemangioma through unenhanced CT images, with an accuracy of 0.817 [37]. This study also uses support vector machine classifier to classify liver malignant tumors and liver hemangioma.
At present, many studies have been done on the classification of focal liver lesions. The strength of our study lies in its comprehensive analysis, which not only included patient clinical characteristics such as sex and age but also lesion characteristics such as size, average CT value, and location—factors often overlooked in previous studies. And we use AutoML, which aims to automate the tedious and repetitive tasks. This allows data researchers to build highly scalable, efficient and high-performance models without sacrificing the quality of the models. We also compared the diagnostic performance of our model against that of radiologists when analyzing the same lesions. A significant advantage of our study is the integration of imaging features assessed by a radiologist and the clinical features of the patients.
We found that sex and age were the only factors without a direct correlation to liver disease occurrence, although the incidence of hepatic malignancies was higher in males than in females. Focal liver lesions are more likely to occur in patients over 60 years. Approximately 70% of HC and HH had a maximum diameter of less than 20 mm, whereas approximately 90% of HA and HM exceeded this size. Most HC, HH, HM, and HA occur under the capsule, particularly HM and HA. It is speculated that this is related to the abundant blood vessels on the liver surface, which are conducive to cell growth [38]. Additionally, malignant liver lesions are significantly different from benign lesions because they cause changes in the surface profile of the liver probably owing to their rapid growth and rich blood flow on the liver surface, thus leading to prominent extrahepatic expansion [39].
Another strength of this study is that both the imaging and clinical characteristics of the lesions were summarized by radiologists.
However, this study has limitations. First, Due to the small sample size and retrospective nature of our study, the accuracy of the model needs to be further improved. And there was no further classification of liver malignancies into subtypes such as primary liver cancer, cholangiocarcinoma, or metastatic tumors. In addition, Radiomics relies on large amounts of data and complex machine learning algorithms, which can lead to problems of overfitting and inadequate generalization. We hope to address these issues in future studies. We will include more cases from different hospitals to expand datasets and more types of focal liver lesions, so as to meet the data size and diversity and use prospective data to further validate the model. In addition, an automatic lesion detection model will be built to form a complete diagnostic process of target detection, discovery of lesions, classification and determination of lesion nature.
Conclusion
We developed a radiomics model to distinguish focal liver lesions using non-contrast CT images. This radiomics-based model offers the potential for enhancing clinical diagnosis and provides additional radiological insights through noninvasive methods.
Acknowledgements
The abstract of this study has been accepted as an oral presentation by the European Congress of Radiology 2024.
Abbreviations
- CT
Computed tomography
- MRI
Magnetic resonance imaging
- FNH
Focal nodular hyperplasia
- HC
Hepatic cyst
- HH
Hepatic hemangioma
- HA
Hepatic abscess
- HCC
Hepatocellular carcinoma
- HM
Hepatic malignancy
- ROI
Region of interest
- VOI
Volume of interest
- ICC
Intraclass correlation coefficient
- AutoML
Automated machine learning
- MLJAR
mljar-supervised
- AUC
Area under the curve
- AI
Artificial intelligence
Author contributions
Conceptualization, N.Y., L.J. and M.L.; Methodology, N.Y, L.J., Z.M. and L.Z.; Software, W.J.; Formal analysis, N.Y., Z.M. and L.J.; Investigation, L.J.; Resources, L.J. and M.L.; Data curation, L.J., N.Y. and Z.M.; Writing– original draft, L.J.; Writing– review & editing, L.J.,Q.X. and M.L.; Supervision, L.J. and M.L.
Funding
This work was supported by the National Key Research and Development Program (grant number 2022YFF1203301), Science and Technology Planning Project of the Shanghai Science and Technology Commission (grant number 22Y11910700), National Natural Science Foundation of China (grant number 61976238), and Shanghai “Rising Stars of Medical Talent” Youth Development Program “Outstanding Youth Medical Talents” (SHWJRS [2021]-99), Emerging Talent Program (XXRC2213) and Leading Talent Program (LJRC2202) of Huadong Hospital, and Excellent Academic Leaders of Shanghai (2022XD042). The funding had no role in the study design, collection, analyses, interpretation of data, writing of the manuscript, or decision to publish the results.
Data availability
All data will be shared upon reasonable request from the corresponding authors.
Declarations
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Nan Yang, Zhuangxuan Ma and Ling Zhang contributed equally to this work.
Contributor Information
Qian Xi, Email: xiqian1129@163.com.
Ming Li, Email: ming_li@fudan.edu.cn.
Liang Jin, Email: jin_liang@fudan.edu.cn.
References
- 1.GattI M, Maino C, Tore D, et al. Benign focal liver lesions: The role of magnetic resonance imaging. World J Hepatol, 2022, 14(5): 923– 43,10.4254/wjh.v14.i5.923. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ronot M, Vilgrain V. Imaging of benign hepatocellular lesions: current concepts and recent updates. Clin Res Hepatol Gastroenterol, 2014, 38(6): 681–8,10.1016/j.clinre.2014.01.014. [DOI] [PubMed] [Google Scholar]
- 3.Gore R M, Pickhardt P J, Mortele K J, et al. Management of Incidental Liver Lesions on CT: A White Paper of the ACR Incidental Findings Committee. J Am Coll Radiol, 2017, 14(11): 1429-37,10.1016/j.jacr.2017.07.018. [DOI] [PubMed] [Google Scholar]
- 4.Reizine E, Mulé S, Luciani A. Focal Benign Liver Lesions and Their Diagnostic Pitfalls. Radiol Clin North Am, 2022, 60(5): 755– 73,10.1016/j.rcl.2022.05.005. [DOI] [PubMed] [Google Scholar]
- 5.Mamone G, DI Piazza A, Carollo V, et al. Imaging of primary malignant tumors in non-cirrhotic liver. Diagn Interv Imaging, 2020, 101(9): 519– 35,10.1016/j.diii.2020.01.010. [DOI] [PubMed] [Google Scholar]
- 6.Marya N B, Powers P D, Fujii-Lau L, et al. Application of artificial intelligence using a novel EUS-based convolutional neural network model to identify and distinguish benign and malignant hepatic masses. Gastrointest Endosc, 2021, 93(5),10.1016/j.gie.2020.08.024. [DOI] [PubMed]
- 7.Li T, Qin L-X, Ji Y, et al. Atypical hepatic focal nodular hyperplasia presenting as acute abdomen and misdiagnosed as hepatocellular carcinoma. Hepatol Res, 2007, 37(12): 1100–5, 10.1111/j.1872-034X.2007.00164.x [DOI] [PubMed] [Google Scholar]
- 8.Wang C L, Cohan R H, Ellis J H, et al. Frequency, outcome, and appropriateness of treatment of nonionic iodinated contrast media reactions. AJR Am J Roentgenol, 2008, 191(2): 409– 15,10.2214/AJR.07.3421. [DOI] [PubMed] [Google Scholar]
- 9.Rudnick M R, Leonberg-Yoo A K, Litt H I, et al. The Controversy of Contrast-Induced Nephropathy With Intravenous Contrast: What Is the Risk?. Am J Kidney Dis, 2020, 75(1): 105– 13,10.1053/j.ajkd.2019.05.022. [DOI] [PubMed] [Google Scholar]
- 10.Reinig J W, Dwyer A J, Miller D L, et al. Liver metastasis detection: comparative sensitivities of MR imaging and CT scanning. Radiology, 1987, 162(1 Pt 1): 43–7, 10.1148/radiology.162.1.3786786 [DOI] [PubMed] [Google Scholar]
- 11.Im W H, Song J S, JANG W. Noninvasive staging of liver fibrosis: review of current quantitative CT and MRI-based techniques. Abdom Radiol (NY), 2022, 47(9): 3051–67,10.1007/s00261-021-03181-x. [DOI] [PubMed] [Google Scholar]
- 12.PetitclerC L, Sebastiani G, Gilbert G, et al. Liver fibrosis: Review of current imaging and MRI quantification techniques. J Magn Reson Imaging, 2017, 45(5): 1276-95,10.1002/jmri.25550. [DOI] [PubMed] [Google Scholar]
- 13.Nino-Murcia M, Olcott E W, Jeffrey R B, et al. Focal liver lesions: pattern-based classification scheme for enhancement at arterial phase CT. Radiology, 2000, 215(3): 746–51. 10.1148/radiology.215.3.r00jn03746 [DOI] [PubMed] [Google Scholar]
- 14.Gibbs J F, Litwin A M, Kahlenberg M S. Contemporary management of benign liver tumors. Surg Clin North Am, 2004, 84(2): 463–80, 10.1016/j.suc.2003.11.003 [DOI] [PubMed] [Google Scholar]
- 15.Caseiro-Alves F, Brito J, Araujo A E, et al. Liver haemangioma: common and uncommon findings and how to improve the differential diagnosis. Eur Radiol, 2007, 17(6): 1544–54. 10.1007/s00330-006-0503-z [DOI] [PubMed] [Google Scholar]
- 16.Li H, Zhu Y, Burnside E S, et al. MR Imaging Radiomics Signatures for Predicting the Risk of Breast Cancer Recurrence as Given by Research Versions of MammaPrint, Oncotype DX, and PAM50 Gene Assays. Radiology, 2016, 281(2): 382–91. 10.1148/radiol.2016152110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Toyama Y, Hotta M, Motoi F, et al. Prognostic value of FDG-PET radiomics with machine learning in pancreatic cancer. Sci Rep, 2020, 10(1): 17024,10.1038/s41598-020-73237-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Huang Y, Liu Z, HE L, et al. Radiomics Signature: A Potential Biomarker for the Prediction of Disease-Free Survival in Early-Stage (I or II) Non-Small Cell Lung Cancer. Radiology, 2016, 281(3): 947–57. 10.1148/radiol.2016152234 [DOI] [PubMed] [Google Scholar]
- 19.Wei J, Jiang H, Gu D, et al. Radiomics in liver diseases: Current progress and future opportunities. Liver Int, 2020, 40(9): 2050–63,10.1111/liv.14555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Grazioli L, Ambrosini R, Frittoli B, et al. Primary benign liver lesions. Eur J Radiol, 2017, 95: 378–98,10.1016/j.ejrad.2017.08.028. [DOI] [PubMed] [Google Scholar]
- 21.Harding-Theobald E, Louissaint J, Maraj B, et al. Systematic review: radiomics for the diagnosis and prognosis of hepatocellular carcinoma. Aliment Pharmacol Ther, 2021, 54(7): 890–901,10.1111/apt.16563. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sia D, Villanueva A, Friedman S L, et al. Liver Cancer Cell of Origin, Molecular Class, and Effects on Patient Prognosis. Gastroenterology, 2017, 152(4): 745– 61,10.1053/j.gastro.2016.11.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Mavilia M G, Molina M, Wu G Y. The Evolving Nature of Hepatic Abscess: A Review. J Clin Transl Hepatol, 2016, 4(2): 158– 68,10.14218/JCTH.2016.00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Namasivayam S, Martin D R, Saini S. Imaging of liver metastases: MRI. Cancer Imaging, 2007, 7(1): 2–9. 10.1102/1470-7330.2007.0002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Lardière-Deguelte S, Ragot E, Amroun K, et al. Hepatic abscess: Diagnosis and management. J Visc Surg, 2015, 152(4): 231– 43,10.1016/j.jviscsurg.2015.01.013. [DOI] [PubMed] [Google Scholar]
- 26.Loh S, Bagheri S, Katzberg R W, et al. Delayed adverse reaction to contrast-enhanced CT: a prospective single-center study comparison to control group without enhancement. Radiology, 2010, 255(3): 764– 71,10.1148/radiol.10091848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Llovet J M, Zucman-RossI J, Pikarsksy E, et al. Hepatocellular carcinoma. Nat Rev Dis Primers, 2016, 2: 16018,10.1038/nrdp.2016.18. [DOI] [PubMed] [Google Scholar]
- 28.AlshareF A, Aggarwal K, SONIA, et al. Review of ML and AutoML Solutions to Forecast Time-Series Data. Arch Comput Methods Eng, 2022, 29(7): 5297– 311,10.1007/s11831-022-09765-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Wever M, Tornede A, Mohr F, et al. AutoML for Multi-Label Classification: Overview and Empirical Evaluation. IEEE Trans Pattern Anal Mach Intell, 2021, 43(9): 3037–54,10.1109/TPAMI.2021.3051276. [DOI] [PubMed] [Google Scholar]
- 30.Burti S, Zotti A, Contiero B, et al. Computed tomography features for differentiating malignant and benign focal liver lesions in dogs: A meta-analysis. Vet J, 2021, 278: 105773,10.1016/j.tvjl.2021.105773. [DOI] [PubMed] [Google Scholar]
- 31.Kelley R K, Bridgewater J, Gores G J, et al. Systemic therapies for intrahepatic cholangiocarcinoma. J Hepatol, 2020, 72(2): 353– 63,10.1016/j.jhep.2019.10.009. [DOI] [PubMed] [Google Scholar]
- 32.Davenport M S, Perazella M A, Yee J, et al. Use of Intravenous Iodinated Contrast Media in Patients with Kidney Disease: Consensus Statements from the American College of Radiology and the National Kidney Foundation. Radiology, 2020, 294(3): 660-8,10.1148/radiol.2019192094. [DOI] [PubMed] [Google Scholar]
- 33.Gillies R J, Kinahan P E, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology, 2016, 278(2): 563–77,10.1148/radiol.2015151169. [DOI] [PMC free article] [PubMed]
- 34.Li Y, Huang X, Xia Y, et al. Value of radiomics in differential diagnosis of chromophobe renal cell carcinoma and renal oncocytoma. Abdom Radiol (NY), 2020, 45(10): 3193– 201,10.1007/s00261-019-02269-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Hu S, Lyu X, Li W, et al. Radiomics Analysis on Noncontrast CT for Distinguishing Hepatic Hemangioma (HH) and Hepatocellular Carcinoma (HCC). Contrast Media Mol Imaging, 2022, 2022: 7693631,10.1155/2022/7693631. [DOI] [PMC free article] [PubMed]
- 36.Zhao X, Liang P, Yong L, et al. Radiomics Study for Differentiating Focal Hepatic Lesions Based on Unenhanced CT Images. Front Oncol, 2022, 12: 650797,10.3389/fonc.2022.650797. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Huang Y-L, Chen J-H, Shen W-C. Diagnosis of hepatic tumors with texture analysis in nonenhanced computed tomography images. Acad Radiol, 2006, 13(6): 713–20. 10.1016/j.acra.2005.07.014 [DOI] [PubMed] [Google Scholar]
- 38.KendalL T, Verheij J, Gaudio E, et al. Anatomical, histomorphological and molecular classification of cholangiocarcinoma. Liver Int, 2019, 39 Suppl 1,10.1111/liv.14093. [DOI] [PubMed] [Google Scholar]
- 39.Milette S, Sicklick J K, Lowy A M, et al. Molecular Pathways: Targeting the Microenvironment of Liver Metastases. Clin Cancer Res, 2017, 23(21): 6390–9,10.1158/1078-0432.CCR-15-1636. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All data will be shared upon reasonable request from the corresponding authors.