MRI radiomics model differentiates small hepatic metastases and abscesses in periampullary cancer patients

Abstract

This multi-center, retrospective study focused on periampullary cancer patients undergoing MRI for hepatic metastasis and abscess differentiation. T1-weighted, T2-weighted, and arterial phase images were utilized to create radiomics models. In the training-set, 112 lesions in 54 patients (median age [IQR, interquartile range], 73 [63–80]; 38 men) were analyzed, and 123 lesions in 55 patients (72 [66–78]; 34 men) comprised the validation set. The T1-weighted + T2-weighted radiomics model showed the highest AUC (0.82, 95% CI 0.75–0.89) in the validation set. Notably, < 30% T1-T2 size discrepancy in MRI findings predicted metastasis (Ps ≤ 0.037), albeit with AUCs of 0.64–0.68 for hepatic metastasis. The radiomics model enhanced radiologists’ performance (AUCs, 0.85–0.87 vs. 0.80–0.84) and significantly increased diagnostic confidence (P < 0.001). Although the performance increase lacked statistical significance (P = 0.104–0.281), the radiomics model proved valuable in differentiating small hepatic lesions and enhancing diagnostic confidence. This study highlights the potential of MRI-based radiomics in improving accuracy and confidence in the diagnosis of periampullary cancer-related hepatic lesions.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-74311-w.

Introduction

The liver is one of the most common sites for cancer metastasis, accounting for nearly 25% of all cases¹, making hepatic metastasis more common than primary hepatic tumors². Hepatic metastasis exhibits distinctive imaging findings such as rim enhancement, centripetal enhancement and capsular retraction adjacent to the lesion³. Meanwhile, hepatic abscess is a localized collection of necrotic inflammatory materials caused by bacterial, fungal or parasitic infection⁴ that commonly presents with imaging findings including the “double target sign” and “cluster sign”^5–7. Despite these characteristic imaging findings, the two diseases also have imaging findings in common such as peripheral rim enhancement in the arterial phase (AP)^8–11 and hyperintense rim in diffusion weighted imaging (DWI)¹². With this overlap, the differentiation of hepatic metastasis and abscess through imaging still remains a challenging task. However, an accurate diagnosis is crucial as the treatment strategies for these two diseases are completely different.

In many cases, magnetic resonance imaging (MRI) is indispensable when evaluating hepatic lesions as it offers more information through contrast-enhancement kinetics and advanced imaging techniques such as DWI¹³. Compared to fluoro-2-deoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT), MRI demonstrated highest sensitivity (93.1%) and improved specificity (87.3%), which was better than CT (73.5%) and comparable to FDG-PET (93.9%)⁸. This is because MRI sequences such as DWI and hepatobiliary phase (HBP) are particularly useful for identifying small lesions, especially those with a diameter of less than 1 cm⁷.

Currently, several key MRI findings for differentiating hepatic metastasis and abscess are known, but their relevance have been inconsistent. Some studies have reported persistent rim enhancement from the arterial phase (AP) to the transitional phase (TP) as favoring abscess over metastasis^8,14,15. However, Kim et al. reported size discrepancy between T1-weighted image (T1-WI) and HBP¹⁶ while Zhuo et al. reported peripheral high signal intensity (SI) rim on DWI and difference in margin sharpness between TP and HBP as the only significant imaging findings suggesting hepatic abscess¹⁷. These inconsistent results suggest that there is yet to be a reliable MR imaging finding for differentiating hepatic metastasis and abscess that can be clinically applied. In addition, an accurate diagnosis is even more difficult when a lesion is small in size or a patient exhibits atypical clinical symptoms. Moreover, patients with a history of malignant biliary obstruction or a bile duct surgery are equally prone to developing either hepatic metastasis or abscess⁸. Under these circumstances, radiomics, which provides nonvisual information by extracting quantitative features from imaging^18,19, may be a better tool for differentiating hepatic metastasis and abscess compared to traditional visual assessment. Previously, there has been one study each that investigated the feasibility of CT- and ultrasound-based radiomics models for differentiating focal liver lesions, including both malignant and nonmalignant tumors^20,21. However, no study has yet investigated the use of an MRI-based radiomics model to differentiate hepatic metastasis from abscess.

Therefore, this study aimed to build a radiomics model using MRI sequences as potential imaging markers to differentiate small (< 2 cm) hepatic metastasis and abscess in patients with periampullary cancer. Meanwhile, MR images were also qualitatively assessed by radiologists to identify significant MR findings that could differentiate hepatic metastasis from abscess. Then, the diagnostic performance of these significant MR findings and the radiomics model was compared, with an evaluation of reader confidence in interpretation before and after the use of the radiomics model.

Results

Patient characteristics

Of 109 patients, 54 patients were in the training set (n = 112; median age [IQR, interquartile range], 73 [63–80]; 38 men) and 55 patients were in the validation set (n = 123; median age, 72 [66–78]; 34 men) (Table 1). Tumor factors such as hepatic lesion size, number of hepatic lesions per patient, and proportion of the hepatic lesion that was pathologically confirmed did not differ between the two sets. Patient characteristics are compared between patients with hepatic metastasis and patients with abscess in the training and validation set in Additional File 1, Supplementary Table S1.

Table 1.

Comparison of baseline characteristics between the training and validation set.

	Training-set (hospital 1) (n = 112, 54 patients)	Validation-set (hospital 2) (n = 123, 55 patients)	P-value
Age, years	73.0 (63.0, 80.0)	72.0 (66.0, 77.5)	0.882
Gender (M/F)	38 (70.4%)/16 (29.6%)	34 (61.8%)/21 (38.2%)	0.459
Body mass index (BMI)	21.8 (20.1, 24.1)	21.8 (20.2, 23.7)	0.669
Type of primary cancer			0.020
AoV cancer	9 (16.7%)	8 (14.5%)
CBD cancer	18 (33.3%)	12 (21.8%)
Duodenal cancer	6 (11.1%)	2 (3.6%)
Pancreatic head cancer	21 (38.9%)	33 (60.0%)
Type of MRI study			0.269
Dynamic MRI (ECA)	9 (16.7%)	15 (27.3%)
Dynamic MRI (HBA)	45 (83.3%)	40 (72.7%)
Laboratory findings
C-reactive protein, mg/L	15.6 (4.0, 50.0)	17.9 (3.1, 44.5)	0.495
WBC count, (103)/uL	6.8 (5.1, 8.9)	6.5 (5.2, 8.2)	0.544
Total bilirubin, mg/dL	0.7 (0.5, 1.3)	0.5 (0.4, 1.4)	0.235
Direct bilirubin, mg/dL	0.2 (0.1, 0.8)	0.5 (0.2, 2.0)	0.022
AST, IU/L	33.5 (24.0,59.0)	29.0 (20.5, 64.5)	0.285
ALT, IU/L	32.0 (18.0,80.0)	30.0 (17.5, 61.5)	0.384
CA19-9, U/mL	98.0 (18.2,444.2)	77.6 (15.8, 1077.0)	0.916
Hepatic metastases / hepatic abscess	67 (59.8%) / 45 (40.2%)	83 (67.5%)/40 (32.5%)	0.222
Interval between MRI acquisition and laboratory findings (days)	1.0 (1.0, 9.0)	2.0 (1.0, 9.0)	0.466
Size of hepatic lesion (mm)	13.7 (10.7, 16.6)	12.6 (10.6, 15.0)	0.111
Number of hepatic lesions per patient	2.0 (1.0, 3.0)	2.0 (1.0, 3.0)	0.597
Pathologic confirmation of hepatic lesion (patients)	11 (20.4%)	17 (30.9%)	0.298
Days since date of MRI to pathologic confirmation of liver lesion	11.0 (3.0, 15.0)	5.0 (1.5,10.0)	0.360

Data are presented as medians (25th -75th percentiles) or numbers (%). AoV ampulla of Vatar, ALT alanine transaminase, AST aspartate transaminase, CA 19 − 9 carbohydrate antigen 19 − 9, CBD common bile duct, ECA extracellular agent, HBA hepatobiliary agent, MRI magnetic resonance imaging, WBC white blood cell.

MRI radiomics analysis

Radiomics features with interobserver ICC > 0.75 were selected for modeling, and 692, 682, and 385 features from unenhanced T1-WI, T2-WI and AP images of the training-set, respectively, met this criterion. After hierarchical clustering, 57, 58, 49, 55, 66, 55 and 68 features were obtained for unenhanced T1-WI, T2-WI, AP, T1-WI + AP, T1-WI + T2-WI, T2-WI + AP, and T1-WI + T2-WI + AP models, respectively.

The performances of the single and merged models for predicting hepatic metastasis in the training and validation sets are summarized in Table 2. In the training set, the T2-WI + AP and unenhanced T1-WI + T2-WI + AP models showed the same best AUROCs of 0.99 (95% CI 0.97, 1.00) and 0.99 (95% CI 0.98, 1.00), respectively, followed by the unenhanced T1-WI + T2-WI model, which showed an AUROC of 0.97 (95% CI 0.95, 1.00). In the validation set, the unenhanced T1-WI + T2-WI model showed the best AUROC of 0.82 (95% CI 0.75, 0.89), and this was significantly better than the AUROCs of other models (Ps ≤ 0.021). The receiver operating characteristics curves of the radiomics models for predicting hepatic metastasis in the training and validation sets are presented in Fig. 1. Images from a representative case are presented in Fig. 2.

Table 2.

Sensitivity, specificity, PPV, NPV and accuracy of the radiomics model for predicting hepatic metastasis.

	Model	Sensitivity, 95%CI (%)	Specificity, 95% CI (%)	PPV, 95% CI (%)	NPV, 95% CI (%)	AUROC, 95% CI	P-value*
Training set	Unenhanced-T1-WI model	83.8 (72.9, 91.6)	86.8 (76.4, 93.8)	86.4 (77.4, 92.2)	84.3 (75.6, 90.3)	0.94 (0.91, 0.98)	0.166
	T2-WI model	70.6 (58.3, 81.0)	92.7 (83.7, 97.6)	90.6 (80.3, 95.8)	75.9 (68.4, 82.1)	0.93 (0.88, 0.97)	0.265
	AP model	88.3 (78.1, 94.8)	85.3 (74.6, 92.7)	85.7 (77.1, 91.5)	87.9 (79.0, 93.3)	0.96 (0.92, 0.99)	0.502
	Unenhanced-T1-WI + AP model	91.2 (81.8, 96.7)	97.1 (89.8, 99.6)	96.9 (88.8, 99.2)	91.7 (83.7, 95.9)	0.98 (0.96, 1.00)	0.724
	Unenhanced-T1-WI + T2-WI model	92.6 (83.7, 97.6)	91.2 (81.8, 96.7)	91.3 (83.0, 95.8)	92.5 (84.2, 96.7)	0.97 (0.95, 1.00)	0.48
	T2-WI + AP model	95.6 (87.6, 99.1)	94.1 (85.6, 98.4)	94.2 (86.3, 97.7)	95.5 (87.6, 98.5)	0.99 (0.97, 1.00)	(Ref.)
	Unenhanced-T1-WI + T2-WI + AP model	98.5 (92.1, 99.9)	95.6 (87.6, 99.1)	95.7 (88.1, 98.5)	98.5 (90.3, 99.8)	0.99 (0.98, 1.00)	0.999
Validation set	Unenhanced-T1-WI model	88.0 (79.0, 94.1)	47.5 (31.5, 63.9)	77.7 (71.9, 82.5)	65.5 (49.4, 78.7)	0.78 (0.69, 0.86)	0.016
	T2-WI model	37.4 (26.9, 48.7)	82.5 (67.2, 99.7)	81.6 (68.2, 90.2)	38.8 (33.8, 44.1)	0.68 (0.58, 0.77)	0.021
	AP model	80.7 (70.6, 88.6)	22.5 (10.8, 38.5)	68.4 (64.0, 72.5)	36.0 (21.4, 53.7)	0.53 (0.42, 0.64)	< 0.001
	Unenhanced-T1-WI + AP model	86.8 (77.5, 93.2)	30.0 (16.6, 46.5)	72.0 (67.4, 76.2)	52.2 (34.6, 69.3)	0.69 (0.60, 0.79)	< 0.001
	Unenhanced-T1-WI + T2-WI model	88.0 (79.0, 94.1)	60.0 (43.3, 75.1)	82.0 (75.6, 87.1)	70.6 (56.0, 81.9)	0.82 (0.75, 0.89)	(Ref.)
	T2-WI + AP model	56.6 (45.3, 67.5)	65.0 (48.3, 79.4)	77.1 (67.9, 84.2)	41.9 (34.1, 50.2)	0.64 (0.53, 0.74)	0.003
	Unenhanced-T1-WI + T2-WI + AP model	89.2 (80.4, 94.9)	37.5 (22.7, 54.2)	74.8 (69.7, 79.2)	62.5 (44.4, 77.7)	0.76 (0.67, 0.84)	< 0.001

AUROC area under the receiver operating characteristic curve, PPV positive predictive value, NPV negative predictive value.

*P-value was calculated using the DeLong test based on comparison to the reference model with best AUROC.

Fig. 1.

Receiver operating characteristic curve analysis of various radiomics models in the (A) training set and (B) validation set for predicting hepatic metastasis.

Fig. 2.

HBA-MR images ((A) unenhanced T1-WI; (B) arterial phase; (C) portal venous phase; (D) transitional phase; (E) hepatobiliary phase (HBP); (F) fat-saturated T2-WI; (G) diffusion weighted image (DWI) b 800; (H) unenhanced T1-WI with segmentation mask) of an 82-year-old male patient with pancreatic head cancer. There is a 12.9 mm sized single focal lesion in the right hepatic lobe. This lesion shows a non-persistent rim enhancement limited to the arterial phase and a subtle HBP low SI rim, without T1-HBP or T1-T2 size discrepancy. Both radiologists interpreted this lesion as hepatic metastasis while the radiomics model predicted it as hepatic abscess. The lesion, however, was clinically confirmed as hepatic abscess based on its disappearance at follow-up.

Visual assessment of MR imaging parameters known to distinguish hepatic metastases from abscesses

The qualitative analysis results of MR imaging parameters observed for hepatic metastases and abscesses are summarized in Additional File 1, Supplementary Table S2. Interobserver agreement for all imaging parameters was excellent (k = 0.716–0.933) (Additional File 1, Supplementary Table S3). Three parameters showed significant differences between hepatic metastases and abscess according to both readers. HBP low SI rim was more present in hepatic metastases than abscesses. In contrast, ≥ 30% unenhanced T1-T2 size discrepancy and ≥ 30% unenhanced T1-HBP size discrepancy were more present in hepatic abscesses than metastases. These three parameters were also significant in the univariable regression analyses (Table 3). However, in the multivariable analyses, only < 30% unenhanced T1-T2 size discrepancy was a significant predictor of hepatic metastasis (reader 1, OR 0.197 [95% CI 0.043, 0.903], P = 0.037; reader 2, OR 0.179 [95% CI 0.038, 0.840], P = 0.029).

Table 3.

Significant MRI findings for differentiating hepatic metastasis and abscess based on univariable and multivariable logistic regression analyses.

		Hepatic metastasis (n = 83, 39 patients)	Hepatic abscess (n = 40, 16 patients)	Univariable logistic regression		Multivariable logistic regression
				OR (95% CI)	P-value	OR (95% CI)	P-value
Reader 1
HBP low SI rim*	Absence	42 (61.8%)	19 (86.4%)	1 (Ref.)	1 (Ref.)	1 (Ref.)	1 (Ref.)
	Presence	26 (38.2%)	3 (13.6%)	3.921 (1.056–14.563)	0.041	3.155 (0.747–13.330)	0.118
T1-T2 size discrepancy	< 30%	79 (95.2%)	27 (67.5%)	1 (Ref.)	1 (Ref.)	1 (Ref.)	1 (Ref.)
	≥ 30%	4 ( 4.8%)	13 (32.5%)	0.105 (0.032–0.350)	< 0.001	0.197 (0.043–0.903)	0.037
T1-HBP size discrepancy*	< 30%	60 (88.2%)	11 (50.0%)	1 (Ref.)	1 (Ref.)	1 (Ref.)	1 (Ref.)
	≥ 30%	8 (11.8%)	11 (50.0%)	0.133 (0.044–0.407)	< 0.001	0.302 (0.080–1.136)	0.076
Reader 2
HBP low SI rim*	Absence	44 (64.7%)	20 (90.9%)	1 (Ref.)	1 (Ref.)	1 (Ref.)	1 (Ref.)
	Presence	24 (35.3%)	2 (9.1%)	5.455 (1.174–25.349)	0.030	3.019 (0.607–15.020)	0.177
T1-T2 size discrepancy	< 30%	79 (95.2%)	24 (60.0%)	1 (Ref.)	1 (Ref.)	1 (Ref.)	1 (Ref.)
	≥ 30%	4 ( 4.8%)	16 (40.0%)	0.076 (0.023–0.249)	< 0.001	0.179 (0.038–0.840)	0.029
T1-HBP size discrepancy*	< 30%	59 (86.8%)	12 (54.5%)	1 (Ref.)	1 (Ref.)	1 (Ref.)	1 (Ref.)
	≥ 30%	9 (13.2%)	10 (45.5%)	0.183 (0.061–0.546)	0.002	0.508 (0.129–2.006)	0.334

*Applied to only focal hepatic lesions observed in multiphasic MRI using HBA. CI confidence interval, HBA hepatobiliary agent, HBP hepatobiliary phase, OR odds ratio, SI signal intensity.

Comparison of radiologist interpretation before and after using the radiomics model

Since the unenhanced T1-WI + T2-WI radiomics model showed the best AUROC in the validation set, radiologist performance for predicting hepatic metastasis with and without this model was compared. In addition, as < 30% unenhanced T1-T2 size discrepancy was a visually assessable MRI finding significantly predictive of hepatic metastasis in the multivariable analysis, its performance for predicting hepatic metastasis was also evaluated to compare it to that of the radiomics model.

Based on the < 30% unenhanced T1-T2 size discrepancy criterion, the sensitivity and specificity for hepatic metastasis was 95.2% (95% CI 88.1, 98.7) and 32.5% (95% CI 18.6, 49.1) for reader 1, and 95.2% (95% CI 88.1, 98.7) and 40.0% (95% CI 24.9, 56.7) for reader 2 (Table 4). The sensitivity of radiologists ranged from 92.8% (95% CI 84.9, 97.3) to 96.4% (95% CI 89.8, 99.3) and specificity ranged from 67.5% (95% CI 50.9, 81.4) to 72.5% (95% CI 56.1, 85.4), respectively, for predicting hepatic metastasis without the radiomics model. When both readers were equipped with the radiomics model, sensitivity ranged from 92.8% (95% CI 84.9, 97.3) to 98.8% (95% CI 93.5, 99.9) and specificity ranged from 75.0 (95% CI 58.8, 87.3) to 77.5% (95% CI 61.5, 89.2). Although AUROCs with the radiomics model (0.85–0.87) was higher than without (0.80–0.84), the difference was not significant (Ps ≤ 0.281).

Table 4.

Comparison of diagnostic performances of a significant visually assessable MRI finding from multivariable regression analysis (< 30% T1-T2 size discrepancy), and radiologist interpretations with and without the radiomics model (unenhanced T1-WI + T2-WI) for predicting hepatic metastasis.

	< 30% T1-T2 size discrepancy1	Radiologist2	Using Radiomics Model3	P-value12	P-value13	P-value23
Reader 1
Sensitivity, 95% CI (%)	95.2 (88.1, 98.7)	92.8 (84.9, 97.3)	92.8 (84.9, 97.3)	0.480	0.527	0.999
Specificity, 95% CI (%)	32.5 (18.6, 49.1)	67.5 (50.9, 81.4)	77.5 (61.5, 89.2)	< 0.001	< 0.001	0.248
PPV, 95% CI (%)	74.5 (65.1, 82.5)	85.6 (79.1, 90.3)	89.5 (82.8, 93.9)	< 0.001	< 0.001	0.251
NPV, 95% CI (%)	76.5 (50.1, 93.2)	81.8 (66.9, 90.9)	83.8 (70.1, 91.9)	0.615	0.527	0.778
AUROC, 95% CI	0.64 (0.55, 0.72)	0.80 (0.72, 0.87)	0.85 (0.78, 0.91)	0.001	< 0.001	0.281
Reader 2
Sensitivity, 95% CI (%)	95.2 (88.1, 98.7)	96.4 (89.8, 99.3)	98.8 (93.5, 99.9)	0.654	0.180	0.157
Specificity, 95% CI (%)	40.0 (24.9, 56.7)	72.5 (56.1, 85.4)	75.0 (58.8, 87.3)	< 0.001	0.005	0.317
PPV, 95% CI (%)	76.7 (67.3, 84.5)	87.9 (81.5, 92.3)	89.1 (82.7, 93.4)	0.007	0.003	0.222
NPV, 95% CI (%)	80.0 (56.3, 94.3)	90.6 (75.8, 96.8)	96.8 (80.9, 99.5)	0.291	0.106	0.151
AUROC, 95% CI	0.68 (0.59, 0.76)	0.84 (0.77, 0.90)	0.87 (0.80, 0.92)	0.004	< 0.001	0.104

P-value¹² compared diagnostic performance between <30% T1-T2 size discrepancy¹ and radiologist not using radiomics model²; P-value¹³ compared diagnostic performance between <30% T1-T2 size discrepancy¹ and radiologist using radiomics model³; P-value²³ compared diagnostic performance between radiologist not usingradiomics model² and radiologist using radiomics model³.

Comparison of radiologist confidence with and without the radiomics model

Both readers showed increased confidence in their interpretations with the radiomics model (Fig. 3). When lesions were divided by confidence scores into two groups (≤ 3 and ≥ 4 points), the overall number of lesions with confidence scores ≥ 4 significantly increased with the radiomics model (reader 1, 88% vs. 76%, P < 0.001; reader 2, 87% vs. 66%, P < 0.001).

Fig. 3.

Distribution of reader 1 and 2 confidence scores with and without the use of the radiomics model (unenhanced T1-WI + T2-WI) for (A) all focal hepatic lesions based on a 5-point scoring system, (B) all focal hepatic lesions divided into two groups based on confidence scores (≤ 3 or ≥ 4 points), (C) hepatic metastases divided only based on confidence scores (≤ 3 or ≥ 4 points), and (D) hepatic abscess divided only based on confidence scores (≤ 3 or ≥ 4 points).

Discussion

In this study, we demonstrate that a radiomics model built using unenhanced T1-WI and T2-WI can be used to predict hepatic metastasis with a high AUROC of 0.99 in the training set and 0.82 in the validation set. Despite differences in primary cancer and MRI acquisition protocol between the training and validation sets, the high performance of this model reflects its robustness and general applicability. Our results show that, although radiologists predicted hepatic metastases more accurately with the radiomics model, this improvement did not reach statistical significance. Nevertheless, the radiologists’ overall confidence in their interpretations significantly improved, and this was especially the case for hepatic metastasis than abscess.

Depending on the study patient or type of MRI exam performed, different visually assessable MRI findings, including persistent rim enhancement from AP to TP, size discrepancy between T1-WI and HBP, size discrepancy between T1-WI and T2-WI, peripheral high SI rim on DWI, and margin conspicuity difference between T1-WI and HBP, have been suggested as distinctive to hepatic metastasis or abscess^8,16,17,22. In our study, T1-T2 size discrepancy was the only significant visually assessable MRI finding that could differentiate hepatic metastasis and abscess. To note, our study does not refute past conclusions as T1-HBP size discrepancy and AP rim persistent through DP or TP were more prevalent in hepatic abscess than metastasis in our study as well, although these findings were not significant in the multivariable analyses. Furthermore, HBP low SI rim was significantly more prevalent in hepatic metastasis in our study and this result was consistent with those of previous studies^23–25. While these visually assessable MRI findings can be categorized to either favor hepatic metastasis or abscess, in many cases, a single lesion can present with several findings together, and even more confusingly, findings favoring metastasis such as non-persistent rim enhancement may coexist with findings favoring abscess such as ≥ 30% size discrepancy between T1-WI and T2-WI. This makes an accurate diagnosis challenging for radiologists who base their interpretations on visual assessment.

Under these circumstances, the radiomics model may be useful in increasing diagnostic accuracy and reader confidence. Consistent with the results of our multivariable analysis, the unenhanced T1-WI + T2-WI radiomics model showed the highest performance for differentiating hepatic metastasis from abscess. Adding the radiomics signatures of AP to one of the unenhanced T1-WI, T2-WI and T1-WI + T2-WI models only lowered the specificity of the model with marginal increase in sensitivity, and this finding is consistent with the fact that peripheral rim enhancement in AP is known to be a nonspecific imaging finding observed in benign and malignant lesions alike, which can include both hepatic abscess and metastasis^8–11. As for the best radiomics model using unenhanced T1-WI and T2-WI, while the T1-T2 size discrepancy may have contributed to the radiomics model, it was certainly not the only factor since T1-T2 size discrepancy alone showed significantly inferior AUROC and specificity for hepatic metastasis. In addition, when the use of contrasts or long MRI processes are not feasible to patients, the radiomics model coupled with abbreviated MRI without contrast could be a viable option when evaluating hepatic metastasis and abscess.

Admittedly, despite its high sensitivity, the low specificity of the unenhanced T1-WI + T2-WI radiomics model may explain why the increase in radiologist performance with the radiomics model was marginal. Such low specificity may increase the false-positive diagnosis of hepatic metastasis, but the use of the radiomics model is never intended as a confirmatory diagnosis and we do not expect it to replace the need for subsequent biopsy. Rather, based on the understanding that a radiomics model makes decisions solely based on the numeric feature set from high-dimensional medical data regardless of clinical situations, radiologists should utilize the radiomics model as a complementary tool to possibly reduce the underdiagnosis of hepatic metastasis in patients with a history of periampullary cancer. Our results show that the radiomics model increases the radiologists’ confidence in diagnosing hepatic metastasis without decreasing specificity. In this regard, complementary information provided by the radiomics model may assist qualitative assessments made by radiologists.

This study had several limitations. First, there may have been a selection bias due to its retrospective nature. Second, only patients with a history of periampullary cancer were included, which may have introduced a selection bias. However, hepatic abscess tends to develop in these patients due to biliary obstruction or an upstream infection caused by biliary stent or post-operative anastomosis. These patients are more often in challenging situations that require differentiation between hepatic abscess and metastasis, making them the cases most necessitating the use of such a radiomics model. Third, not all focal hepatic lesions were pathologically confirmed and for those that were not, we tried to include only strongly suspected abscesses and metastases based on the aforementioned clinical diagnosis criterion. Lastly, the MRI scans were obtained from various MRI scanners with different field strengths (1.5T and 3.0T), vendors, and acquisition protocols, and this may have potentially influenced the performance of the prediction model. However, the high performance of the radiomics model despite the use of heterogenous MRI can be interpreted as a reflection of its robustness and generalizability.

The MRI-based radiomics model showed better performance for differentiating small hepatic metastasis and abscess than that of the most relevant visually assessable MRI finding, which was T1-T2 size discrepancy. When used by radiologists, the MRI-based radiomics model helped increase reader confidence in interpretations.

Methods

Study patients

This retrospective study was conducted in accordance to the principles stated in the Declaration of Helsinki and approved by the institutional review board of Yonsei Health System (IRB No. 4-2022-1191), and the need to obtain informed consent was waived owing to its retrospective nature. In addition, this study adhered to the CLEAR (CheckList for EvaluAtion of Radiomics) reporting guidelines to improve the credibility, reproducibility, and transparency of the study²⁶.

We queried the radiological database for patients who had been diagnosed with periampullary cancer and undergone multiphasic contrast-enhanced MRI with the terms “abscess” or “metastasis” included in radiological MR reports uploaded between October 2009 and December 2019 at hospital 1 (Gangnam Severance Hospital) and between October 2005 and December 2019 at hospital 2 (Severance Hospital). The exclusion criteria for patients were as follows: (1) no focal liver lesions; (2) size of focal hepatic lesions > 2 cm; (3) size of focal hepatic lesions ≤ 5 mm or presenting as tiny diffusion restricting foci without detection on dynamic phases in abdominal MRI; and (4) focal hepatic lesions without follow-up CT or MRI. For the hospital 2 cohort, patients whose fat-saturated T2 weighted images (FS-T2WI) were obtained via echo time (TE) < 80ms and ≥ 85ms were also excluded to closely resemble the TE (83-85ms) of the FS-T2WI protocol of hospital 1. In addition, patients at hospital 2 whose focal liver lesions (1) had typical MRI findings for cystic lesions; (2) had previously received treatment; and (3) were obscured due to MR artifacts were also excluded. Finally, 54 patients with 67 hepatic metastases and 45 hepatic abscesses were allocated from hospital 1 to the training set while 55 patients with 83 hepatic metastases and 40 hepatic abscesses were allocated from hospital 2 to the validation set (Fig. 4). 11 (20.4%) out of 54 patients in the training set and 17 (30.9%) out of 55 patients in the validation set had focal hepatic lesions pathologically confirmed. The rest were confirmed via clinical diagnosis based on follow-up ≥ 6 months with CT or MRI, wherein hepatic abscess was clinically diagnosed if the lesion disappeared or decreased in size with antibiotic treatment at follow-up, and hepatic metastasis was clinically diagnosed if (1) the size of the lesion did not change or increase despite antibiotic treatment, or (2) the lesion persisted, regardless of size increase or decrease, after chemotherapy.

Fig. 4.

Flowchart summarizing patient selection and allocation to the (A) training and (B) validation sets.

Sample size calculation

In a previous study, the prevalence of a focal hepatic lesion being metastasis rather than abscess in periampullary cancer patient was 56.9% (41/72)⁸. To calculate the sample size, we used the method for calculating the sample size required for a clinical prediction model based on the accuracy of estimate²⁷. In brief, the required sample size (n) can be calculating using the anticipated outcome proportion (φ), and the desired margin of error () using the following equation:

 

Using the above prevalence of 0.569 as the anticipated outcome proportion and setting the margin of error at 0.1, the minimum required sample size was calculated to be 95 patients. In the current study, 109 patients were included, exceeding the minimum sample size, which allows us to develop a reliable prediction model.

MRI acquisition

Multiphasic contrast-enhanced MRI was performed with either a 1.5-T or 3.0-T MRI scanner. Routine protocols included T1-weighted three-dimensional gradient-echo imaging with dynamic contrast enhancement, respiratory triggered or breath-hold T2-weighted imaging (WI) with or without fat suppression, and diffusion-weighted imaging (DWI). Details in MR parameters and protocols are provided in Additional File 1, Supplementary Table S4.

MR radiomics analysis

Image segmentation

One radiologist (J.H.P.) used 3D Slicer version 4.10 (www.slicer.org), a free and open-source software, to semi-automatically segment the entire area of focal liver lesions on three-dimensional, gradient-echo axial T1-WI, T2-WI, and AP images for feature extraction. Segmentation masks from unenhanced T1-WI was registered to T2-WI and AP images. All segmentation masks were confirmed by a senior radiologist (J.C.), and disagreements were resolved in consensus. Both radiologists were blinded to clinical and histopathologic data during this process. Another board-certified radiologist (J.Y.) independently performed tumor segmentation on the training set to analyze interobserver reproducibility.

Radiomics feature extraction

Prior to feature extraction, the Synthetic Minority Oversampling Technique (SMOTE) algorithm was used to address class imbalance between the number of hepatic metastases and abscesses by generating synthetic data based on the existing five nearest neighbors. Radiomics features were extracted separately for unenhanced T1-WI, T2-WI and AP images using PyRadiomics, an open-source Python package (version 2.1.2; https://pyradiomics.readthedocs.io), via radiomics.featureextractor.RadiomicsFeatureExtractor class²⁸. The following settings were used as arguments: {‘binWidth’: 20, ‘resampledPixelSpacing’: [3, 3, 3], ‘interpolator’: sitk.sitkBSpline, ‘normalize’: True, ‘normalizeScale’: 1, ‘removeOutliers’: True, ‘sigma’: [-3, 3]}. All image types were enabled, including original, wavelet, Laplacian of Gaussian filter, square, square root, logarithm, exponential, gradient, local binary pattern 2D, and local binary pattern 3D. Features were normalized for each MRI sequence using the z-score normalization method. A total of 873 features were extracted for each MRI sequence.

Feature selection and building the classification model

Only radiomics features with good interobserver reproducibility (intraclass correlation coefficient [ICC] > 0.75) were included in the analysis and 692, 682, and 385 features from unenhanced T1-WI, T2-WI and AP images, respectively, in the training set met this criterion. Subsequently, the least absolute shrinkage and selection operator (LASSO) method was used to select the most useful predictive features from the training set. In the LASSO method, 10-fold cross-validation was used to select the optimal regularization parameter alpha, as the average mean squared error of each patient was the smallest. With the optimal alpha, features having a nonzero coefficient in LASSO were considered robust predictors. The support vector machine²⁹ with a linear kernel was used to construct models using features from one of or combinations of unenhanced T1-WI, T2-WI and AP images.

Implementation

All codes were written and run on Google Colab (https://colab.research.google.com, n.d.), which provides 12GB of RAM and an NVIDIA Tesla K80 GPU. Python 3.10.4 was used along with the Python libraries NumPy, Pandas, Scikit-learn, and EDSR. Codes used for radiomics modeling and data analysis have been deposited into a publicly accessible repository (https://github.com/jhp0510/Metastasis-vs.-Abscess---Radiomics-).

MRI visual assessment

Two board-certified abdominal radiologists (H.R. and Y.E.C. with 9 and 16 years of experience, respectively) retrospectively and independently reviewed the images in the validation set. One radiologist (J.H.P) measured the size of each liver lesion and recorded its section number 2 weeks prior to image analysis. Both readers were blinded to the clinical or histopathological results of each case. MRIs of both groups were presented randomly in a blinded manner to avoid bias.

For qualitative analysis, the following imaging parameters were evaluated: (a) rim enhancement on each phase including DP for ECA-MRI and TP and HBP for HBA-MRI, (b) low signal intensity (SI) rim on HBP for HBA-MRI and (c) DWI pattern at b value of 800–900 s/mm², which was subdivided into three categories: (1) homogenous high SI relative to liver parenchyma through the whole area of the focal hepatic lesion, (2) high SI rim confined to the periphery of the focal hepatic lesion with relatively lower SI in the center, and (3) high SI confined only to the center of the focal hepatic lesion. In addition, size discrepancies, which were defined as ≥ 30% difference in the longest diameter of the lesion between unenhanced T1-WI and T2-WI as well as unenhanced T1-WI and HBP were also analyzed. These imaging parameters were selected because previous studies reported them as for differentiating hepatic metastases and abscesses^8,16,17,22. Interobserver agreement was assessed for MR imaging parameters after the first independent image analysis. In addition to the above parameters, two readers scored the final interpretation (hepatic metastasis vs. abscess) before and after using the radiomics model. Both readers also evaluated their confidence in diagnoses with a 5-point scoring system (1-least confident; 5-highly confident) before and after using the radiomics model.

Study outcomes

The primary outcome was the diagnostic performance, specifically the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, for various MRI-based radiomics models constructed using different combinations of T1-WI, T2-WI, and AP images. The secondary outcomes included the diagnostic performance of the most relevant visually assessable MRI findings from multivariable regression analysis and the performance of radiologists with and without the use of the best-performing MRI-based radiomics model.

Statistical analyses

The Shapiro-Wilk test was used to assess normality. Non-normally distributed continuous variables are presented as medians (interquartile range, IQR). Continuous variables were compared using the Mann-Whitney U test while categorical variables were assessed using the X²-test or Fisher’s exact test. The ICC was calculated to evaluate interobserver agreement between radiologists and interobserver reproducibility for radiomics features: an ICC > 0.75 was considered to indicate good reproducibility³⁰. Interobserver agreement between radiologists was expressed by Cohen’s kappa coefficient. The diagnostic performances of the radiomics model and radiologists for predicting hepatic metastases were calculated and the AUCs of receiver operating characteristics curve were compared using the DeLong method. Statistical analyses were performed using R version 3.4.3 (R Foundation for Statistical Computing). Two-sided P < 0.05 was considered statistically significant.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Author contributions

JHP and YE are responsible for design conception. JHP collected data. JHP, EC, JY, and HR analyzed and interpreted the data, JHP, JP, JC and YEC participated in manuscript writing and editing. All authors read and approve the final manuscript.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.

Ethical approval

This retrospective study was conducted in accordance to the principles stated in the Declaration of Helsinki and approved by the institutional review board of Severance Hospital (IRB No. 4-2022-1191), and the need to obtain informed consent was waived owing to its retrospective nature.