Large Language Models for Preoperative Microvascular Invasion Prediction in Hepatocellular Carcinoma: A Multicenter Comparison with Radiologists and Treatment Outcomes

Abstract

Background

Preoperative prediction of microvascular invasion (MVI) in hepatocellular carcinoma (HCC) is critical for prognosis but challenging. This study evaluated the performance of large language models (LLMs) for MVI prediction compared with radiologists with varying levels of experience and explored the association of model-predicted MVI with recurrence and treatment outcomes.

Materials and Methods

In this retrospective multicenter study, 602 patients with pathologically confirmed HCC who underwent preoperative MRI and surgical resection were included. GPT-4o and DeepSeek-R1, prompted in English and Chinese using radiology report narratives and predefined MRI features, were compared with six radiologists using histopathology as the reference standard. An independent cohort of 135 patients who underwent radiofrequency ablation (RFA) was included for treatment-outcome analyses. Prediction performance was assessed using diagnostic accuracy and the area under the curve (AUC).

Results

GPT-4o (English input) achieved 75.6% accuracy, outperforming residents (59.3%, P < 0.001) and attendings (66.7%, P < 0.001), and was comparable to senior radiologists (74.2%, P = 1.000). DeepSeek-R1 (Chinese input) achieved the highest accuracy (81.2%), outperforming all radiologists (P < 0.05). Residents and attendings showed high sensitivity but relatively low specificity, whereas senior radiologists demonstrated a more balanced performance; while DeepSeek-R1 showed better specificity (up to 94.2%, P < 0.001). LLMs performance were relatively stable across tumor sizes, whereas accuracy among less-experienced radiologists improved with tumor size. AUCs for LLMs (GPT-4o: 75.4%; DeepSeek-R1 Chinese: 80.1%) were superior or comparable to those of radiologists (61.8%–74.5%). Stacking ensemble-predicted MVI-positive status was associated with shorter time-to-recurrence (TTR) in both small (≤3 cm, P = 0.039) and large (>3 cm, P = 0.001) tumors, and with better outcomes following anatomical rather than non-anatomical resection (≤3 cm, P = 0.021; >3 cm, P < 0.001). After propensity score matching, surgical resection was associated with longer TTR than RFA in the model-predicted MVI-positive subgroup, but not in the MVI-negative subgroup.

Conclusion

LLMs, particularly DeepSeek-R1 and GPT-4o, demonstrated diagnostic performance for preoperative MVI prediction comparable to or better than that of senior radiologists, with relatively stable performance across tumor sizes and potential value for recurrence-risk stratification and treatment planning.

Background

Hepatocellular carcinoma (HCC) is a leading cause of cancer-related mortality, with recurrence rates reaching up to 70% within 5 years despite advances in treatment.¹ Microvascular invasion (MVI), a marker of aggressive tumor behavior, significantly influences recurrence risk.² For patients at high MVI risk, more aggressive surgical strategies, such as wider resection margins, can be employed to reduce recurrence risk.^3,4 Accurate preoperative MVI identification enables tailored surgical strategies, including anatomical resection or wider resection margins, and informs the use of neoadjuvant therapies like immune checkpoint inhibitors and anti-angiogenic agents, ultimately improving surgical outcomes and long-term survival.^5,6

Magnetic resonance imaging (MRI) is widely used for MVI prediction, with features such as non-smooth margins, arterial peritumoral enhancement, and hepatobiliary phase hypointensity demonstrating predictive value.^7–9 The Liver Imaging Reporting and Data System (LI-RADS) provides standardized criteria to enhance reporting consistency.¹⁰ However, radiology-based MVI prediction faces challenges due to variability in radiologist expertise, subjective interpretation, and non-standardized reporting, often resulting in incomplete or inconsistent assessment of critical imaging features.^11,12 These limitations underscore the need for innovative tools to improve objectivity and reproducibility in preoperative evaluation.

Large language models (LLMs), such as GPT-4o and DeepSeek-R1, have shown remarkable capabilities in natural language processing.^13–15 Recent studies demonstrate their potential to extract and interpret clinically relevant information from radiology reports, including automated report generation, disease classification, and identification of imaging findings.^16–20 Emerging research also explores LLMs’ ability to analyze nuanced semantic information from large-scale datasets, enhancing patient outcome predictions and clinical decision support.^21,22 Nevertheless, their application to preoperative MVI prediction in HCC, particularly through analyzing the full textual content of radiology reports, remains underexplored. Building on these insights, we hypothesized that an LLMs-based approach evaluated using bilingual radiology reports (Chinese and English) could provide more objective and consistent MVI predictions compared to radiologists of varying experience levels. Moreover, integrating LLMs-derived predictions with survival analysis could offer clinically actionable insights for personalized HCC treatment strategies.

Therefore, this study aimed to compare the diagnostic performance of two mainstream LLMs and radiologists across different experience levels, revealing their potential impact on clinical decision-making.

Materials and Methods

Study Design and Population

This retrospective multicenter study included patients with pathologically confirmed single HCC who underwent surgical resection between January 2019 and December 2023 at two tertiary centers (Army Medical University and The Third Affiliated Hospital of Chongqing Medical University). A total of 602 eligible patients were retrospectively identified and included, with 277 classified as MVI-positive and 325 as MVI-negative. Inclusion criteria comprised: (a) pathologically proven HCC following curative liver resection, with clearly documented MVI status; (b) Age ≥ 18 years; (c) Barcelona Clinic Liver Cancer (BCLC) stage 0 or A; and (d) Availability of complete clinical and laboratory data. Exclusion criteria comprised: (a) Absence of preoperative MRI or an interval of more than one month between MRI examination and surgery; (b) severe imaging artifacts; and (c) Receipt of any preoperative anticancer treatment, including transarterial chemoembolization, ablation, radiotherapy, or systemic therapy. Figure 1 presented the flowchart of the experimental design.

Figure 1.

Flowchart of the study design. From January 2019 to December 2023, 1503 patients with pathologically confirmed hepatocellular carcinoma (HCC) who underwent curative liver resection at two tertiary care hospitals were screened. After applying the inclusion and exclusion criteria, 602 patients with available preoperative MRI and complete clinical, radiological, and pathological data were enrolled. Patients were stratified according to pathological microvascular invasion (MVI) status (MVI-positive, n = 277; MVI-negative, n = 325) and tumor size (≤2 cm, n = 93; >2 to ≤3 cm, n = 125; >3 to ≤5 cm, n = 213; and >5 cm, n = 171). Original radiology report content and seven predefined MRI features were used as inputs for large language models (LLMs; ChatGPT and DeepSeek) in both English and Chinese. The LLMs and six radiologists with varying levels of liver imaging experience (two residents, two attending radiologists, and two senior radiologists) independently predicted MVI, using histopathological findings as the reference standard. The association between stacking ensemble-predicted MVI status and treatment outcomes was further evaluated.

In addition to the 602 patients surgical resection cohort, an independent treatment-comparison cohort consisting of 135 patients with solitary HCC measuring ≤3 cm who underwent radiofrequency ablation (RFA) during the same study period was included. Because pathological MVI status was unavailable in the absence of surgical specimens, the RFA-treated patients were not included in the diagnostic performance analyses and were not evaluated by the full six-radiologist panel; however, the two senior radiologists included in the stacking ensemble independently assessed these patients to provide radiologist inputs for ensemble prediction. Instead, the RFA cohort was included exclusively in the treatment-outcome analyses comparing TTR after RFA versus surgical resection in patients with tumors ≤ 3 cm.

This study was conducted in accordance with the guidelines of the Declaration of Helsinki and was approved by the Institutional Review Board of the Army Medical University (approval No. KY2023107). The requirement for written informed consent was waived by the Ethics Committee of the Army Medical University because of the retrospective nature of the study. All patient data were de-identified before analysis, and patient confidentiality was strictly maintained throughout data collection, processing, analysis, and reporting in accordance with institutional data-protection requirements.

MRI Scanning Protocol

MRI examinations were performed on 3.0T scanners (Trio Tim (Siemens Healthineers) and Discovery MR750W (GE Healthcare)). The imaging protocol included axial and coronal T2-weighted sequences, in-phase and opposed-phase T1-weighted imaging, and dynamic contrast-enhanced imaging with gadoxetate disodium (Gd-EOB-DTPA) administered via the antecubital vein at 1.0 mL/s and 0.1 mL/kg body weight. Arterial phase acquisition commenced when the contrast agent reached the abdominal aorta, guided by real-time monitoring, with patients instructed to hold their breath. Portal venous phase imaging occurred 55–70 seconds post-injection, followed by the transitional phase at 150–180 seconds. Hepatobiliary phase (HBP) imaging was obtained 15 minutes post-contrast administration. Although MRI examinations were performed on two 3.0-T systems from different vendors, all centers used a common sequence framework and the same weight-based gadoxetate disodium protocol. Because the LLMs did not directly process raw images, image-level preprocessing, including resampling and signal-intensity normalization, was not performed.

Imaging Feature Assessment and Postoperative Follow-up

Because this was a retrospective study, all preoperative MRI examinations from both the surgical resection and RFA cohorts were re-reviewed specifically for the present analysis. The study-specific imaging assessments were not extracted from the diagnostic conclusions of the original clinical MRI reports. Seven predefined MRI features associated with MVI risk were independently evaluated for each tumor by two abdominal radiologists (F.X.C. and X.M.L., with 10 and 12 years of experience in abdominal imaging, respectively), without access to histopathological MVI status. In cases of disagreement, the final decision was made by a senior abdominal radiologist (P.C., with 30 years of experience). The imaging features and their definitions are provided in Supplementary Table S1. The consensus imaging features were subsequently incorporated into the MRI report text for downstream MVI assessment.

Postoperative surveillance involved imaging (ultrasound, contrast-enhanced CT, or MRI) every 3–6 months for the first two years, then every 6–12 months thereafter. Tumor recurrence was identified by characteristic imaging findings, including arterial phase hyperenhancement with portal venous or delayed phase washout. Time-to-recurrence (TTR) was measured from the date of the index treatment (surgical resection or RFA, as applicable) to recurrence, with follow-up ending on July 1, 2025.

Ground Truth and MVI Status

Six readers, stratified by experience-two residents (postgraduate years 2–3), two attending radiologists (6–10 years of practice), and two senior radiologists (≥15 years of subspecialty experience) independently assessed MVI status based on the MRI report text incorporating the seven predefined imaging features, without re-reviewing the original MRI images. Each radiologist provided a binary MVI assessment (positive or negative) without access to histopathological information.

Histopathologic MVI status served as the reference standard. Archived pathological slides were independently re-reviewed by two experienced liver pathologists who were blinded to model predictions and study-specific radiologist interpretations. Disagreements were resolved by consensus. MVI was defined as the presence of tumor cell nests within an endothelium-lined vascular lumen on light microscopy.

Model Configuration and Prompt Design

LLMs inputs combined two complementary sources of textual imaging information: the narrative content of the preoperative MRI reports and seven predefined MRI features manually assessed during study-specific image re-review. Scanner type, center identity, and sequence-specific acquisition parameters were not included in the prompts.

GPT-4o (OpenAI) and DeepSeek-R1 (DeepSeek Tech) were independently evaluated using both English and Chinese inputs. The same standardized zero-shot, single-step prompt template was used for both models and languages, with only the prompt language and model-specific API interface differing. The prompt comprised five modules-context, objective, style, tone, and findings-and required a structured diagnostic summary followed by a binary MVI classification (Figure 2). No chain-of-thought instructions, reference cases, or few-shot examples were provided. All models were run with standardized parameter settings to improve output consistency (temperature = 0.1, max_tokens = 1024, top_p = 0.9, frequency_penalty = 0), with no sampling-based randomness or reinforcement learning introduced. To reduce bias and ensure reproducibility, all prompt executions were performed using automated scripts in a controlled computing environment. Outputs were parsed via predefined, rule-based methods to standardized predictions across models and language settings.

Figure 2.

Complete workflow of MVI prediction using large language models, where original radiology reports are integrated with seven structured MRI features and translated into both English and Chinese. These text reports are then formatted through prompt engineering and separately input into ChatGPT and DeepSeek for structured diagnostic conclusion generation and MVI classification.

Statistical Analysis

Statistical analyses were performed using IBM SPSS Statistics (version 26.0; IBM Corp., Armonk, NY) and Python (version 3.9; Python Software Foundation). Diagnostic performance of the four LLMs configurations and three radiologist experience levels was evaluated in the 602 patient surgical cohort using accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1 score, and area under the curve (AUC), with 95% confidence intervals (CI) calculated using the Wilson method. Overall diagnostic accuracy was the primary endpoint. Pairwise comparisons of diagnostic accuracy were adjusted for multiple testing using the Bonferroni correction.

A stacking ensemble integrating predictions from the four LLMs configurations and two senior radiologists was developed using 5-fold cross-validation and evaluated in a 30% hold-out validation set within the surgical cohort. The validated ensemble model was subsequently applied to the independent cohort of 135 RFA-treated patients to generate model-predicted MVI status; these patients were not included in the diagnostic accuracy analyses because pathological MVI status was unavailable.

Treatment-outcome analyses comparing surgical resection with RFA were restricted to patients with tumors ≤3 cm. Propensity score matching (PSM) was used exclusively for these comparisons. Separate 1:1 nearest-neighbor matching without replacement (caliper, 0.2) was performed for the overall cohort and the model-predicted MVI-negative and MVI-positive subgroups, using age, sex, platelet count, white blood cell count, albumin, total bilirubin, tumor size, BCLC stage, alpha-fetoprotein, Child–Pugh class, and HBV DNA status as covariates.

TTR was estimated using the Kaplan–Meier method. In PSM resection-versus-RFA cohorts, TTR was compared using a stratified Log rank test with matched pair as the stratum; conventional Log rank tests were used for unmatched survival comparisons, including anatomical versus non-anatomical resection. A two-sided P < 0.05 was considered statistically significant.

Results

Diagnostic Accuracy of LLMs Relative to Radiologists Across Experience Levels

Supplementary Table S2 presents a comparison of baseline characteristics between the two centers. Table 1 summarizes the diagnostic performance of LLMs and radiologists, with GPT-4o (English input) as the reference for multiple comparisons. GPT-4o (English input) achieved 75.6% accuracy (455/602; 95% CI, 72.0–78.9%), with no significant differences from GPT-4o (Chinese input) (75.2%; adjusted P = 1.000) or DeepSeek-R1 (English input) (74.4%; adjusted P = 1.000). DeepSeek-R1 (Chinese input) achieved the highest accuracy (81.2%; 489/602; 95% CI, 77.9–84.1%) and significantly outperformed GPT-4o (English input) (adjusted P = 0.023).

Table 1.

Performance Comparison of GPT-4o (English Input) versus GPT-4o (Chinese Input), DeepSeek-R1 (English and Chinese Inputs), and Radiologists of Varying Experience in Predicting Microvascular Invasion in Hepatocellular Carcinoma

Reader	Total (n = 602)
	Diagnostic Accuracy Rate (%)	χ2	P Value*	Adj P-Value**
Resident 1	58.0 (54.0,61.9) [349/602]	46.049	< 0.001	< 0.001
Resident 2	60.6 (56.6,64.4) [365/602]	36.161	< 0.001	< 0.001
Radiology Resident average	59.3 (56.5,62.0) [357/602]	82.085	< 0.001	< 0.001
Attending 1	67.3 (63.5,70.9) [405/602]	12.626	< 0.001	0.005
Attending 2	66.1 (62.2,69.8) [398/602]	18.566	< 0.001	< 0.001
Attending Radiologist average	66.7 (64.0,69.3) [401.5/602]	30.694	< 0.001	< 0.001
Senior 1	73.3 (69.6,76.7) [441/602]	1.556	0.212	1.000
Senior 2	75.1 (71.5,78.4) [452/602]	0.066	0.798	1.000
Associate Senior Radiologist average	74.2 (71.7,76.6) [446.5/602]	1.099	0.295	1.000
Deepseek_R1_ch	81.2 (77.9,84.1) [489/602]	9.633	0.002	0.023
Deepseek_R1_en	74.4 (70.8,77.7) [448/602]	0.426	0.514	1.000
ChatGPT_ch	75.2 (71.6,78.5) [453/602]	0.036	0.849	1.000
ChatGPT_en	75.6 (72.0,78.9) [455/602]	-	-	-

Notes: Data in parentheses represent 95% confidence intervals (CIs), while those in brackets indicate numerators and denominators. *We used the McNemar χ²-test to compare the diagnostic accuracy of GPT-4o (English input) with each comparator. **Adj P-value were adjusted for multiple comparisons using the Bonferroni correction.

Supplementary analyses using GPT-4o (Chinese input) and DeepSeek-R1 (English input) as reference configurations showed similar patterns: both significantly outperformed residents and attending radiologists but did not differ significantly from senior radiologists (Supplementary Tables S3 and S4). In contrast, DeepSeek-R1 (Chinese input) significantly outperformed all other LLMs configurations and radiologist groups (all adjusted P < 0.05; Supplementary Table S5).

Subgroup Diagnostic Accuracy in MVI-Positive and MVI-Negative Cases

Performance differed substantially between MVI-positive and MVI-negative cases (Table 2). In MVI-positive cases, residents and attendings showed higher sensitivity than GPT-4o (English input) (92.6% and 87.2% vs 72.6%; both adjusted P < 0.001), whereas their specificity in MVI-negative cases was lower (30.9%-49.2% vs 78.2%; adjusted P < 0.001). Senior radiologists showed more balanced performance (79.1% for MVI-positive and 70.0% for MVI-negative cases).

Table 2.

Subgroup Comparison of Diagnostic Accuracy for Microvascular Invasion-Positive and -Negative in Hepatocellular Carcinoma: GPT-4o (English Input) versus GPT-4o (Chinese Input), DeepSeek-R1 (English and Chinese), and Radiologists of Varying Experience Levels

Reader	MVI-Positive			MVI-Negative
	(n = 277)			(n = 325)
	Sensitivity (%)	P Value*	Adj P-Value**	Specificity (%)	P Value*	Adj P-Value**
	95% CI [n/N]			95% CI [n/N]
Resident 1	93.1 (89.5,95.5) [258/277]	< 0.001	< 0.001	28.0 (23.4,33.1) [91/325]	< 0.001	< 0.001
Resident 2	92.1 (88.3,94.7) [255/277]	< 0.001	< 0.001	33.8 (28.9,39.1) [110/325]	< 0.001	< 0.001
Radiology Resident average	92.6 (90.1,94.5) [256.5/277]	< 0.001	< 0.001	30.9 (27.5,34.6) [100.5/325]	< 0.001	< 0.001
Attending 1	85.9 (81.3,89.5) [238/277]	< 0.001	< 0.001	51.4 (46.0,56.8) [167/325]	< 0.001	< 0.001
Attending 2	88.4 (84.1,91.7) [245/277]	< 0.001	< 0.001	47.1 (41.7,52.5) [153/325]	< 0.001	< 0.001
Attending Radiologist average	87.2 (84.2,89.7) [241.5/277]	< 0.001	< 0.001	49.2 (45.4,53.0) [160/325]	< 0.001	< 0.001
Senior 1	80.5 (75.4,84.7) [223/277]	0.002	0.022	67.1 (61.8,72.0) [218/325]	< 0.001	< 0.001
Senior 2	77.6 (72.3,82.1) [215/277]	0.061	0.736	72.9 (67.8,77.4) [237/325]	0.059	0.707
Associate Senior Radiologist average	79.1 (75.5,82.3) [219/277]	< 0.001	0.006	70.0 (66.4,73.4) [227.5/325]	< 0.001	< 0.001
Deepseek_R1_ch	66.1 (60.3,71.4) [183/277]	0.014	0.172	94.2 (91.1,96.3) [306/325]	< 0.001	< 0.001
Deepseek_R1_en	82.3 (77.4,86.3) [228/277]	< 0.001	< 0.001	67.7 (62.4,72.6) [220/325]	< 0.001	0.001
ChatGPT_ch	77.3 (72.0,81.8) [214/277]	0.047	0.569	73.5 (68.4,78.0) [239/325]	0.067	0.802
ChatGPT_en	72.6 (67.1,77.5) [201/277]	-	-	78.2 (73.4,82.3) [254/325]	-	-

Notes: Data in parentheses represent 95% confidence intervals (CIs), while those in brackets indicate numerators and denominators. *We used the McNemar χ²-test to compare the diagnostic accuracy of GPT-4o (English input) with each comparator. **Adj P-value were adjusted for multiple comparisons using the Bonferroni correction.

Among the LLMs, DeepSeek-R1 (English input) achieved the highest performance in MVI-positive cases (82.3%), whereas DeepSeek-R1 (Chinese input) performed best in MVI-negative cases (94.2%). Detailed pairwise comparisons are provided in Supplementary Tables S6–S8. In the MVI-positive subgroup, residents achieved the highest accuracy (92.6%), significantly exceeding all LLMs (all adjusted P < 0.001). In the MVI-negative subgroup, DeepSeek-R1 (Chinese input) achieved the highest accuracy (94.2%), significantly outperforming all radiologist groups (30.9%–70.0%).

Diagnostic Accuracy Stratified by Tumor Size

Diagnostic accuracy according to tumor size is summarized in Table 3. Residents showed markedly lower accuracy for tumors ≤ 3 cm than for tumors > 3 cm (38.1% vs 71.4%, P < 0.001), with a similar but smaller difference among attendings (58.0% vs 71.6%, P < 0.001). Senior radiologists showed stable performance across tumor sizes (72.9% vs 74.9%, P = 0.504).

Table 3.

Subgroup Comparison of Diagnostic Accuracy Stratified by Tumor Size (≤ 3cm vs > 3cm): Radiologists and Large Language Models in the Prediction of Microvascular Invasion in Hepatocellular Carcinoma

Reader	Tumor Size ≤ 3cm	Tumor Size > 3cm	χ2	P Value*
	(n = 218)	(n = 384)
	Diagnostic Accuracy Rate (%)	Diagnostic Accuracy Rate (%)
	95% CI [n/N]	95% CI [n/N]
Resident 1	33.0 (27.1, 39.5) [72/218]	72.1 (67.4, 76.4) [277/384]	85.693	< 0.001
Resident 2	43.1 (36.7, 49.7) [94/218]	70.6 (65.9, 74.9) [271/384]	42.765	< 0.001
Radiology Resident average	38.1 (33.7, 42.7) [83/218]	71.4 (68.1, 74.5) [274/384]	126.259	< 0.001
Attending 1	59.6 (53.0, 65.9) [130/218]	71.6 (66.9, 75.9) [275/384]	8.531	0.004
Attending 2	56.4 (49.8, 62.8) [123/218]	71.6 (66.9, 75.9) [275/384]	13.656	< 0.001
Attending Radiologist average	58.0 (53.3, 62.5) [126.5/218]	71.6 (68.3, 74.7) [275/384]	22.506	< 0.001
Senior 1	69.3 (62.9, 75.0) [151/218]	75.5 (71.0, 79.5) [290/384]	2.467	0.116
Senior 2	76.6 (70.5, 81.7) [167/218]	74.2 (69.6, 78.3) [285/384]	0.305	0.581
Associate Senior Radiologist average	72.9 (68.5, 76.9) [159/218]	74.9 (71.7, 77.8) [287.5/384]	0.447	0.504
Deepseek_R1_ch	85.3 (80.0, 89.4) [186/218]	78.9 (74.5, 82.7) [303/384]	3.344	0.068
Deepseek_R1_en	70.2 (63.8, 75.9) [153/218]	76.8 (72.3, 80.7) [295/384]	2.881	0.090
ChatGPT_ch	72.0 (65.7, 77.5) [157/218]	77.1 (72.6, 81.0) [296/384]	1.653	0.199
ChatGPT_en	75.7 (69.6, 80.9) [165/218]	75.5 (71.0, 79.5) [290/384]	< 0.001	1.000

Notes: Data in parentheses represent 95% confidence intervals (CIs), while those in brackets indicate numerators and denominators. *We used the Pearson χ² test with Yates’ continuity correction to compare diagnostic accuracy between the ≤3 cm and >3 cm tumor-size groups.

In contrast, LLMs performance was relatively stable between tumors ≤ 3 and > 3 cm: GPT-4o (English input), 75.7% vs 75.5% (P = 1.000); GPT-4o (Chinese input), 72.0% vs 77.1% (P = 0.199); DeepSeek-R1 (English input), 70.2% vs 76.8% (P = 0.090); and DeepSeek-R1 (Chinese input), 85.3% vs 78.9% (P = 0.068).

Further analysis across four tumor-size categories (≤ 2, 2–≤ 3, 3–≤ 5, and > 5 cm) confirmed a clear size-dependent improvement in diagnostic accuracy among less-experienced radiologists, most prominently among residents (P < 0.001), whereas senior radiologists also showed significant variation across the four tumor-size categories (P = 0.001). In contrast, LLMs performance varied less across tumor sizes, with DeepSeek-R1 (Chinese input) achieving the highest accuracy in the smallest tumor group (Supplementary Table S9). These trends are illustrated in Figure 3 and Supplementary Figure S1.

Figure 3.

Line chart compares diagnostic accuracy across tumor size categories among LLMs and radiologists.

Discriminative Performance: AUC, Precision, Recall, and F1 Score

As shown in Table 4, AUC increased with radiologist experience, from 61.8% for residents to 68.2% for attendings and 74.5% for senior radiologists. Residents and attendings showed high recall (0.926 and 0.872, respectively) but relatively low precision (0.533 and 0.594). DeepSeek-R1 (Chinese input) achieved the highest AUC (80.1%; 95% CI: 72.3–87.9%), followed by DeepSeek-R1 (English input) (AUC, 75.0%) and GPT-4o (English and Chinese: 75.4%).

Table 4.

Discriminative Performance of Large Language Models and Radiologists for Predicting Microvascular Invasion in Hepatocellular Carcinoma

Reader	Total (n = 602)
	AUC (%) (95% CI)	Precision	Recall	F1	PPV	NPV
Resident 1	60.6% (54.0%, 67.1%)	0.524	0.931	0.671	0.524	0.827
Resident 2	63.0% (56.1%, 69.8%)	0.543	0.921	0.683	0.543	0.833
Radiology Resident average	61.8% (57.0%, 66.5%)	0.533	0.926	0.677	0.533	0.831
Attending 1	68.7% (61.4%, 75.9%)	0.601	0.859	0.707	0.601	0.811
Attending 2	67.8% (60.5%, 75.0%)	0.588	0.884	0.706	0.588	0.827
Attending Radiologist average	68.2% (63.1%, 73.3%)	0.594	0.872	0.707	0.594	0.818
Senior 1	73.8% (66.5%, 81.1%)	0.676	0.805	0.735	0.676	0.801
Senior 2	75.3% (67.9%, 82.6%)	0.710	0.776	0.741	0.710	0.793
Associate Senior Radiologist average	74.5% (69.4%, 79.7%)	0.692	0.791	0.738	0.692	0.797
Deepseek_R1_ch	80.1% (72.3%, 87.9%)	0.906	0.661	0.764	0.906	0.765
ChatGPT_ch	75.4% (68.0%, 82.7%)	0.713	0.773	0.742	0.713	0.791
ChatGPT_en	75.4% (67.9%, 82.8%)	0.739	0.726	0.732	0.739	0.770
Deepseek_R1_en	75.0% (67.7%, 82.3%)	0.685	0.823	0.748	0.685	0.818

Notes: Data in parentheses represent 95% confidence intervals (CIs).

Abbreviations: PPV, positive predictive value; NPV, negative predictive value; AUC, area under the receiver operating characteristic curve; MVI, microvascular invasion.

Survival by Pathological and Stacking Ensemble–Predicted MVI

The stacking ensemble achieved accuracies of 83.57% in the training set and 82.97% in the validation set. Both pathological and stacking ensemble-predicted MVI positive status were associated with shorter TTR in patients with tumors ≤3 cm (P = 0.035 and 0.039, respectively) and > 3 cm (P = 0.006 and 0.001, respectively; Figure 4A–D). In these patients, anatomical resection achieved significantly longer TTR than non-anatomical resection (pathologically identified: P < 0.001; stacking ensemble-predicted: P = 0.021 and < 0.001) (Figure 4E–L). By contrast, no significant differences were observed among MVI-negative patients (pathologically identified: P = 0.549 and 0.309; stacking ensemble-predicted: P = 0.090 and 0.476).

Figure 4.

Kaplan-Meier survival analyses. Kaplan-Meier survival curves for patients with tumor diameter ≤ 3 cm, stratified by (A) pathological MVI (log-rank, P = 0.035) and (B) stacking ensemble-predicted MVI (log-rank, P = 0.039) demonstrated significant differences in time-to-recurrence (TTR). Similarly, in patients with tumor diameter >3 cm, stratified by (C) pathological MVI (log-rank P = 0.006) and (D) stacking ensemble-predicted MVI (log-rank P = 0.001) consistently discriminated TTR outcomes. In patients with tumor diameter ≤ 3 cm, stratified by (E) pathological MVI-negative (log-rank P = 0.549) and (F) stacking ensemble-predicted MVI-negative (log-rank P = 0.090), no significant difference in TTR was observed between anatomical and non-anatomical resection. In the ≤ 3 cm subgroup with MVI positive, stratification by (G) pathological MVI (log-rank P < 0.001) and (H) stacking ensemble-predicted MVI (log-rank P = 0.021) consistently demonstrated significant survival benefit with anatomical resection. In patients with tumor diameter >3 cm, stratified by (I) pathological MVI-negative (log-rank P = 0.309) and (J) stacking ensemble-predicted MVI-negative (log-rank P = 0.476), no survival difference was noted between anatomical and non-anatomical resection. In the > 3 cm subgroup with MVI positive, stratified by (K) pathological (log-rank P < 0.001) and (L) stacking ensemble-predicted (log-rank P < 0.001) consistently favored anatomical resection. After PSM, (M) surgical resection achieved longer TTR than radiofrequency ablation (RFA) in patients with tumors ≤ 3 cm (log-rank P < 0.001). In stacking ensemble-predicted MVI negative patients, (N) resection and RFA showed no significant difference in TTR (log-rank P = 0.121). In stacking ensemble-predicted MVI positive patients, (O) resection provided significantly prolonged TTR compared with RFA (log-rank P < 0.001).

Post-PSM on the patient’s clinical baseline characteristics (Supplementary Tables S10–S12), surgical resection was associated with longer TTR than RFA in patients with tumors ≤3 cm (P < 0.001). Stratified analyses showed no significant difference in model-predicted MVI-negative patients (P = 0.121), whereas surgical resection was associated with longer TTR in model-predicted MVI-positive patients (P < 0.001) (Figure 4M–O).

Discussion

In this study, GPT-4o and DeepSeek-R1 demonstrated promising performance for preoperative MVI prediction from textual MRI information. Overall, the LLMs outperformed less-experienced radiologists and achieved performance comparable to or better than that of senior radiologists. DeepSeek-R1 with Chinese input showed the highest overall accuracy, whereas LLMs performance was relatively stable across tumor-size groups. Importantly, stacking ensemble-predicted MVI status was also associated with TTR and identified subgroups with different recurrence outcomes according to treatment strategy.

Previous studies have shown that LLMs can match or exceed expert radiologists in complex diagnostic tasks.^23–25 Consistent with these findings, GPT-4o and DeepSeek-R1 outperformed less-experienced radiologists and achieved performance comparable to or better than that of senior radiologists. Among the four LLMs configurations, DeepSeek-R1 with Chinese input achieved the highest overall accuracy. However, this should not be interpreted as a general native-language advantage, as GPT-4o showed similar performance with English and Chinese inputs, while DeepSeek-R1 exhibited different language-related patterns across MVI-status and tumor-size subgroups. These findings suggest that language effects may be model- and task-specific, potentially reflecting interactions among model architecture, pretraining data, prompt wording, report phrasing, and case composition. Similar model-dependent differences between DeepSeek-R1 and GPT-based models have been reported in other medical diagnostic settings.^26,27 Because no formal model-by-language interaction or reasoning-mode ablation analysis was performed, the mechanisms underlying these differences remain uncertain and require further validation. Beyond diagnostic performance, the open-source nature of DeepSeek-R1 may facilitate institution-specific fine-tuning and local deployment, potentially reducing the need to transmit sensitive patient data to external servers.²⁸

In this study, radiologists, particularly residents and attendings, exhibited high sensitivity but low specificity (eg, 30.9% for residents), which may reflect a tendency to prioritize the detection of MVI-positive cases.^29,30 In contrast, LLMs showed a more balanced performance, leveraging statistical optimization without medico-legal biases that influence human judgment.³¹ As such, they more readily classified ambiguous cases as negative, leading to superior specificity. These differences underscore the divergent diagnostic strategies and risk tolerance between human readers and LLMs.

Our findings showed that diagnostic performance was more strongly influenced by tumor size among less-experienced radiologists, whereas LLMs maintained relatively stable accuracy across tumor-size groups. MVI has been associated with multiple MRI features, including peritumoral HBP hypointensity, peritumoral arterial enhancement, irregular margins or incomplete capsule, and satellite nodules, with peritumoral HBP hypointensity showing a particularly strong association with histopathologically confirmed MVI.^8,32,33 The greater size dependence observed among less-experienced radiologists may reflect greater reliance on conspicuous tumor characteristics, whereas experienced radiologists and LLMs may incorporate a broader range of MVI-related imaging descriptors. Notably, LLMs also performed well in tumors ≤2 cm, suggesting potential value for MVI assessment in small HCCs, where imaging findings may be less conspicuous.^34,35

Beyond diagnostic performance, the prognostic relevance of model-predicted MVI may have important implications for clinical decision-making. Previous imaging,^33,36,37 radiomics,^38,39 and deep learning-based studies^40,41 have consistently identified MVI as an adverse prognostic factor in HCC, and our findings extend this evidence by showing that stacking ensemble-derived MVI stratification is also associated with recurrence outcomes. Notably, differences in TTR across resection types and treatment modalities were predominantly observed in patients predicted to have MVI, consistent with previous evidence that MVI status may help identify patients with differential outcomes across treatment strategies.^36,42–46 However, given the retrospective design and potential residual confounding despite PSM, these treatment-related associations should be interpreted cautiously and require prospective validation.

Several limitations warrant consideration. First, the retrospective multicenter design warrants external validation in larger and more diverse cohorts. Second, the LLMs relied solely on textual imaging inputs without incorporating laboratory, genomic, or other clinical data. Third, model performance was evaluated only with English and Chinese inputs and may not generalize to other languages or reporting styles. Fourth, multicenter differences in MRI acquisition were not harmonized at the image level or incorporated into the LLMs prompts. Although imaging features were reassessed using standardized definitions, residual acquisition heterogeneity may have affected feature assessment. Finally, general LLMs may have encountered relevant medical knowledge during pretraining, raising the possibility of a “memory effect”; however, its extent could not be determined because the training corpora are not fully transparent.

In conclusion, LLMs, particularly DeepSeek-R1, demonstrated diagnostic performance comparable to experienced radiologists for preoperative MVI prediction in HCC, with relatively stable performance across tumor sizes. Stacking ensemble-predicted MVI was also associated with recurrence outcomes, suggesting the potential of LLMs as complementary tools for preoperative risk stratification and treatment planning, although further prospective validation is warranted.

Funding Statement

This study was supported by the Boqing Innovation Fund of the First Affiliated Hospital of Army Medical University (No. 2024BQCXJJ-8), the AI Medical Device Innovation Tasks: Unveiling the List and Taking the Lead (No. 2025-AISJK-01008) and Clinical research cultivation projects and technology achievement transformation incubation projects (NO. 2026ZHFHC02).

Data Sharing Statement

The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding authors, Wei Chen (landcw@tmmu.edu.cn) and Xiaoming Li (lxm359261069@tmmu.edu.cn), upon reasonable request.

Ethics Approval and Informed Consent

The study was approved by the Institutional Review Board of the Army Medical University (approval No. KY2023107). The requirement for written informed consent was waived by the Ethics Committee of the Army Medical University because of the retrospective nature of the study.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The authors declare no competing interests.