Comparative evaluation of large language models for text-based diagnostic reasoning in fetal central nervous system MRI

Abstract

Fetal magnetic resonance imaging (MRI) plays an important role in evaluating prenatal central nervous system (CNS) abnormalities, but expert interpretation and counseling remain challenging. Large language models (LLMs) may support text-based diagnostic reasoning, yet their performance in fetal CNS MRI has not been well characterized. The study aimed to compare ChatGPT, Gemini, and DeepSeek in text-based diagnostic reasoning for fetal CNS MRI cases. This retrospective single-center study included 85 fetal MRI cases with postnatal diagnostic confirmation. Each case was converted into a standardized text summary containing MRI findings and essential clinical information, without image input. The 3 LLMs generated ranked differential diagnoses, diagnostic reasoning, management recommendations, prognostic counseling points, and clinical cautions. Anonymized outputs were independently evaluated under blinded conditions for principal diagnosis matching, diagnostic accuracy, clinical reasoning quality, clinical suggestion utility, and communication style and safety. DeepSeek achieved the highest principal diagnosis matching rate (66/85, 77.6%), followed by Gemini (59/85, 69.4%) and ChatGPT (52/85, 61.2%). The overall difference was significant (Cochran’s Q = 6.2553, P = .0438), although no pairwise comparison remained significant after Holm correction. DeepSeek showed the highest diagnostic accuracy and reasoning quality, whereas Gemini performed best in clinical suggestion utility and communication style and safety. Exploratory correlation analyses suggested model-specific associations among evaluation metrics. LLM performance in fetal CNS MRI text-based reasoning was multidimensional and model-dependent. These findings support further supervised evaluation of LLMs as text-based decision-support tools but do not support autonomous clinical diagnosis or counseling.

1. Introduction

Congenital anomalies impose a substantial burden on perinatal health worldwide, and clinical management depends heavily on prenatal diagnoses that are accurate, timely, and useful for counseling.^[1,2] Ultrasonography remains the routine first-line modality for fetal screening; however, fetal magnetic resonance imaging (MRI) has become an important adjunct in selected cases.^[3] Because of its high soft-tissue contrast and multiplanar capability, fetal MRI can characterize complex fetal abnormalities, particularly central nervous system (CNS) abnormalities, in greater detail when ultrasound assessment is limited by skull ossification, fetal position, or acoustic shadowing.^[4–6]

Fetal MRI interpretation is not a simple process of image recognition. It requires subtle imaging findings to be interpreted in relation to gestational-age-specific neurodevelopment, evolving fetal anatomy, and the broader maternal–fetal clinical context. For this reason, fetal MRI diagnosis remains partly subjective and depends on subspecialized multidisciplinary expertise that is not evenly distributed across clinical settings.^[3,5] The shortage and maldistribution of pediatric and fetal neuroradiology expertise therefore create a diagnostic bottleneck, particularly for centers with limited access to specialist consultation.^[7,8]

Large language models (LLMs) have attracted increasing interest as support tools in radiology. Recent studies and reviews suggest that these models may assist with report drafting, impression formulation, information summarization, and diagnostic reasoning when the input is primarily text-based.^[9–11] This is clinically relevant because many multidisciplinary workflows begin not with independent image review by every participant, but with referral letters, preliminary imaging reports, and written case summaries. Text-based diagnostic reasoning is therefore a practical setting in which LLMs may integrate structured radiological findings with clinical information to generate differential diagnoses and management-oriented interpretations. This setting is also relevant to fetal MRI, where artificial intelligence (AI) research has increasingly addressed information extraction and report-based tasks in addition to image analysis.^[12,13]

Based on these considerations, this study selected 3 representative and high-performance LLMs – ChatGPT5.5, Gemini 3.0, and DeepSeek-R1 – for a comprehensive comparative evaluation of their performance in fetal MRI text-based diagnostic reasoning. These 3 models represent distinct technological development trajectories: ChatGPT systems are widely recognized for strong general reasoning performance, Gemini emphasizes multimodal and long-context reasoning, while DeepSeek has been explicitly optimized for advanced reasoning behavior.^[14–16] We aimed to determine whether these models could generate reliable diagnostic reasoning from written fetal MRI findings and whether their outputs remained clinically appropriate when the task extended beyond identification of the primary diagnosis to include differential diagnosis, prenatal management recommendations, prognosis-oriented counseling, and safety-sensitive communication. We hypothesized that all 3 models would show some utility in core diagnostic reasoning, but that their performance would diverge in tasks requiring caution, prioritization, and context-dependent clinical judgment.

2. Materials and methods

2.1. Study design and setting

This retrospective single-center study included 85 fetal MRI cases referred to our hospital between December 2023 and January 2026 after prenatal ultrasonography raised suspicion of a fetal CNS abnormality. The case selection process is summarized in Figure 1. The cohort of fetal CNS MRI cases is presented in Table S1, Supplemental Digital Content 1. The institutional review board approved the study and waived the requirement for written informed consent because only de-identified retrospective data were used (ethical approval number: LC2025-057). The final cohort included a broad range of fetal CNS abnormalities, including ventricular abnormalities, callosal anomalies, posterior fossa malformations, neural tube defects, disorders of cortical development, and other structural brain anomalies. For each fetus, the final diagnosis was determined from postnatal imaging follow-up and/or comprehensive postnatal clinical assessment. This postnatal diagnosis served as the reference standard for subsequent analyses. All MRI examinations were acquired on a 3.0-T scanner using the institutional fetal neuroimaging protocol. We evaluated 3 commercial LLMs through their official web interfaces: GPT-5.5 via ChatGPT Plus (OpenAI), Gemini 3.0 via Google AI Pro (Google), and DeepSeek-R1 via a free DeepSeek web account (DeepSeek). All models were queried in April 2026 using the publicly available web interface versions at that time. Model versions corresponded to the available web releases during the study period and may not reflect application programming interface versioning. Before submission, all case materials were fully anonymized. Each case was queried in a new independent session without prior dialogue, memory, custom instructions, image uploads, original Digital Imaging and Communications in Medicine data, or institutional templates. No personally identifiable information or unnecessary metadata were transmitted to the models. Under blinded conditions, model outputs were subsequently evaluated using diagnosis-matching analysis and a multidimensional scoring framework.

Figure 1.

Flowchart of case selection. Patient selection flowchart based on the Standards for Reporting of Diagnostic Accuracy guidelines. Radiology reports from 200 patients who underwent fetal brain MRI during the study period were retrospectively reviewed. A total of 115 patients were excluded because of inadequate image quality or artifacts affecting assessment, irregular or unavailable developmental follow-up, absence of definite imaging abnormalities, or lack of confirmed final diagnostic assessment. The final cohort comprised 85 fetal CNS MRI cases with clearly described radiological reports and definite diagnoses confirmed by developmental specialists. CNS = central nervous system, LLM = large language model, MRI = magnetic resonance imaging.

2.2. Prompt design and model interrogation

Three board-certified radiologists with fetal imaging experience independently prepared standardized natural-language case summaries from the original fetal MRI reports. A unified template was used to retain key MRI findings and essential non-identifying clinical information. To prevent information leakage, all final diagnostic labels, diagnosis-suggestive wording, and postnatal pathological or genetic information were excluded before submission to the models. The standardized prompt was developed by a multidisciplinary team of fetal imaging and developmental specialists and was applied uniformly to all cases:

“You are a senior fetal imaging specialist with expertise in fetal central nervous system MRI diagnosis. Based on the provided imaging features and clinical information, please complete the following tasks: (1) provide the three most likely diagnoses or differential diagnoses, ranked from most to least likely, and indicate the confidence level for each as high, medium or low; (2) explain your diagnostic reasoning in detail, listing the key MRI features that support the top diagnosis and briefly explaining how they distinguish it from other candidate diagnoses; (3) provide clinical management recommendations, including suggested next prenatal tests, referral pathways or specialist consultations, with justification; (4) summarize key points for prognosis counseling for the fetal family, including potential neurodevelopmental outcomes, risk of associated syndromes and survival-related information; (5) highlight important cautions for clinicians and families when interpreting this information.”

All standardized clinical summaries and prompts were written in Chinese and submitted directly to all 3 models. The English prompt presented in the manuscript is a faithful translation provided for international readers.

2.3. Response evaluation and blinded scoring

All model-generated responses were anonymized after collection and randomly relabeled as answer 1, answer 2, or answer 3, thereby concealing the corresponding model identity from the raters. The full set of responses was then assessed by 3 independent evaluators with experience in fetal imaging and prenatal diagnosis. Inter-rater reliability was estimated with a two-way random-effects intraclass correlation coefficient (ICC) for absolute agreement, ICC(2, 1). To reduce possible order effects, the 3 responses generated for each case were displayed in a randomized sequence. Each response was graded on its own merits, rather than by direct pairwise comparison.

Model performance was assessed according to a prespecified multidimensional evaluation framework. Principal diagnosis matching was treated as a binary outcome, with 0 assigned to an incorrect match and 1 assigned to a correct match. Four principal domains were evaluated using a 5-point Likert scale: diagnostic accuracy, including match of principal diagnosis, quality of diagnosis list, and reasonableness of diagnostic confidence; clinical reasoning quality, including MRI evidence quality, depth of differential analysis, and logical reasoning; clinical recommendation quality, including adherence to prenatal management guidelines, reasonableness of prognostic counseling, and comprehensiveness of multidisciplinary coordination; and communication style and safety, including awareness of safety boundaries, extent of uncertainty disclosure, and patient-friendliness. The scoring criteria for evaluating the responses of LLMs are shown in Table S2, Supplemental Digital Content 2.

For descriptive comparison across multiple evaluation domains, Likert-scale scores were linearly transformed from the original 1 to 5 scale to a standardized 0 to 1 scale using the formula x′ = (x − 1)/4, where x represents the original Likert score and x′ represents the transformed value. This transformation was performed only for scale normalization and visualization purposes and did not alter the relative ranking of observations. Therefore, the subsequent nonparametric analyses based on rank comparisons remained unaffected. Thus, original scores of 1, 2, 3, 4, and 5 corresponded to standardized scores of 0, 0.25, 0.50, 0.75, and 1.00, respectively. Raters completed each evaluation form in a single pass and were instructed not to return to previously assigned scores. Evaluation forms containing missing ratings were regarded as invalid and reentered in full.

2.4. Statistical analysis

All statistical analyses were conducted using SPSS 26.0 software (IBM). Continuous and ordinal outcomes were summarized as mean (M) ± standard deviation, while categorical variables were presented as frequencies (percentages). Because each case was evaluated by all 3 models, comparisons were based on paired or repeated-measures data. Binary principal diagnosis matching was compared using Cochran’s Q test, followed by pairwise McNemar tests with Holm correction. Domain scores derived from Likert-scale ratings and transformed to a 0 to 1 scale were summarized as M ± standard deviation and median values. Overall differences among models were assessed using the Friedman test, followed by pairwise Wilcoxon signed-rank tests with Holm correction when appropriate. Kendall W was reported as the effect size for Friedman tests, and rank-biserial correlation or standardized r was reported for pairwise Wilcoxon comparisons. Two-sided P values < .05 were considered statistically significant. Pearson coefficients were used to compare correlation strength and direction across models within a unified metric space. The correlation coefficient matrix is presented visually in heatmap format. Statistical significance in heatmaps was annotated as *P < .05, **P < .01, and ***P < .001. Inter-rater reliability was assessed using ICC(2, 1).

3. Results

A total of 255 model-generated responses were analyzed, corresponding to 85 cases evaluated by 3 LLMs. Inter-rater agreement was good to excellent across scoring domains, with ICC(2, 1) values ranging from 0.88 to 1.00. No model produced a complete safety refusal or failed to provide the requested ranked differential diagnosis. Safety disclaimers, when present, were retained in the original outputs and evaluated within the communication style and safety domain.

3.1. Comparison of principal diagnostic agreement

The performance differences among the models across the 4 dimensions are shown in Figure 2. DeepSeek achieved the highest principal diagnosis matching rate, correctly identifying 66 of 85 cases (77.65%), followed by Gemini with 59 of 85 cases (69.41%) and ChatGPT with 52 of 85 cases (61.18%). Cochran’s Q test showed a significant overall difference among the 3 models (Q = 6.2553, P = .0438). However, no pairwise comparison remained statistically significant after Holm correction. DeepSeek’s higher matching rate should therefore be interpreted as a descriptive trend rather than definitive evidence of pairwise superiority. Descriptive statistics and statistical comparisons are summarized in Table 1.

Figure 2.

Comparative performance of LLMs in fetal MRI cases across 4 dimensions. GPT = Generative Pre-trained Transformer, LLM = large language model, MRI = magnetic resonance imaging.

Table 1.

Comparative performance of ChatGPT, DeepSeek, and Gemini in fetal MRI text-based diagnostic reasoning.

Panel A. Primary endpoint
Endpoint	ChatGPT, n/N (%)		DeepSeek, n/N (%)		Gemini, n/N (%)			Overall test		DS vs CG		DS vs GM		CG vs GM
Principal diagnosis matching	52/85 (61.18)		66/85 (77.65)		59/85 (69.41)			Cochran’s Q = 6.2553, P = .0438		0.0773 (+16.47%)		0.5304 (+8.24%)		0.5304 (−8.24%)

Panel B. Secondary endpoints
Domain		ChatGPT, mean ± SD (Mdn)		DeepSeek, mean ± SD (Mdn)		Gemini, mean ± SD (Mdn)	Overall test		Kendall W		DS vs CG		DS vs GM		CG vs GM
Diagnostic accuracy		0.5784 ± 0.1802 (0.5833)		0.7510 ± 0.2880 (0.8333)		0.6853 ± 0.2859 (0.7500)	Friedman χ2(2) = 34.1603, P < .001		0.201		0.0001 (r = 0.460)		0.1386 (r = 0.161)		0.0044 (r = 0.332)
Clinical reasoning quality		0.5020 ± 0.2669 (0.5833)		0.7333 ± 0.2976 (0.8333)		0.6539 ± 0.2976 (0.7500)	Friedman χ2(2) = 40.9936, P < .001		0.241		<0.001 (r = 0.526)		0.0663 (r = 0.199)		0.0013 (r = 0.370)
Clinical suggestion utility		0.3167 ± 0.1372 (0.3333)		0.6108 ± 0.1779 (0.5833)		0.6922 ± 0.2109 (0.7500)	Friedman χ2(2) = 88.9845, P < .001		0.523		<0.001 (r = 0.795)		0.0341 (r = 0.230)		<0.001 (r = 0.811)
Communication style and safety		0.4755 ± 0.1048 (0.5000)		0.4066 ± 0.1188 (0.4375)		0.5176 ± 0.0991 (0.5625)	Friedman χ2(2) = 35.2653, P < .001		0.207		0.0023 (r = 0.352)		<0.001 (r = 0.615)		0.0220 (r = 0.249)

For the primary endpoint, overall differences were assessed using Cochran’s Q test, followed by pairwise McNemar tests with Holm correction. For the secondary endpoints, overall differences were assessed using Friedman tests, followed by pairwise Wilcoxon signed-rank tests with Holm correction. Values in pairwise comparison columns are adjusted P values, with percentage-point differences or effect sizes in parentheses.

CG = ChatGPT, DS = DeepSeek, GM = Gemini, Mdn = median, MRI = magnetic resonance imaging, SD = standard deviation.

3.2. Comparison of reasoning quality and response quality

Significant inter-model differences were observed across all 4 evaluated domains: diagnostic accuracy, clinical reasoning quality, clinical suggestion utility, and communication style and safety (all Friedman P < .001). DeepSeek showed the strongest diagnostic performance, with the highest scores for diagnostic accuracy (M = 0.751) and clinical reasoning quality (M = 0.733). It significantly outperformed ChatGPT in both domains (adjusted P < .001 for both comparisons) and maintained a descriptive advantage over Gemini.

By contrast, DeepSeek’s relative performance diminished in downstream clinical translation tasks. For clinical suggestion utility, Gemini achieved the highest score (M = 0.692), significantly outperforming both DeepSeek (M = 0.611; adjusted P = .0341) and ChatGPT (M = 0.317; adjusted P < .001). Gemini also attained the highest score for communication style and safety (M = 0.518), significantly outperforming ChatGPT (adjusted P = .0220) and DeepSeek (adjusted P < .001). Despite its stronger diagnostic performance, DeepSeek had the lowest communication style and safety score.

3.3. Exploratory inter-metric correlation analysis

Exploratory correlation heatmaps showed model-specific patterns of association among evaluation metrics as depicted in Figure 3A–C. Across the 3 models, principal diagnosis matching tended to be positively associated with MRI evidence quality and logical reasoning, although the strength and distribution of these associations varied by model.

Figure 3.

Correlation heatmaps of evaluation metrics for the 3 large language models. (A) ChatGPT correlation heatmap of evaluation metrics; (B) DeepSeek correlation heatmap of evaluation metrics; (C) Gemini correlation heatmap of evaluation metrics. MRI = magnetic resonance imaging.

For DeepSeek, MRI evidence quality, depth of differential analysis, and logical reasoning were positively associated with principal diagnosis matching, suggesting that higher diagnostic agreement was accompanied by higher expert-rated reasoning scores in this dataset. For ChatGPT, the quality of the diagnosis list was negatively correlated with principal diagnosis matching, MRI evidence quality, and logical reasoning, indicating that broader diagnostic lists were not necessarily associated with better diagnostic performance. For Gemini, logical reasoning showed positive associations with adherence to prenatal management guidelines and reasonableness of prognostic counseling. In the communication domain, patient-friendliness was negatively correlated with communication clarity in Gemini. This association should be interpreted descriptively and does not establish that empathetic or patient-centered language caused reduced clarity. Together with the broader observation that patient-friendliness was not consistently associated with core diagnostic accuracy in any model, this suggests that gains in clinical precision and communicative rigor may sometimes occur at the expense of interpersonal warmth.

Finally, uncertainty disclosure followed a consistent pattern across all 3 models. The extent of uncertainty disclosure was broadly negatively correlated with principal diagnosis matching, MRI evidence quality, and logical reasoning, indicating that more frequent expressions of uncertainty were more often associated with limited analytic performance than with calibrated, proactive safety behavior. Overall, the correlation analyses indicate that the 3 models differed not only in absolute performance but also in the internal organization of their abilities. DeepSeek was characterized by a compact diagnostic-reasoning core, Gemini by broader diagnostic-to-management integration, and ChatGPT by greater metric dissociation and weaker alignment between response elaboration and diagnostic correctness.

4. Discussion

This study evaluated the text-based diagnostic reasoning performance of 3 general-purpose LLMs using standardized fetal CNS MRI case summaries. The main finding was that model performance varied across diagnostic agreement, reasoning quality, clinical suggestion utility, and communication safety. This supports the view that medical LLM evaluation should move beyond surface-level benchmarking and toward structural analysis of model behavior.^[17]

In prenatal CNS imaging, written case summaries contain gestational-age-specific anatomy, lesion morphology, associated abnormalities, and clinical context. These elements must be integrated before a ranked diagnosis can be translated into management and counseling. Current fetal MRI guidelines and reviews emphasize that MRI complements expert ultrasound by refining anatomical diagnosis and supporting prognostic assessment, while also recognizing the difficulty of individualized prenatal counseling.^[3–5] In this context, correct principal diagnosis represents only 1 component of useful clinical reasoning. A model that reaches the right diagnostic label but provides weak management advice or unsafe counseling language would still fall short of clinical readiness. The text-only design should not be interpreted only as a limitation. It reflects a common clinical situation in which decision-making is mediated through referral summaries, radiology reports, structured case presentations, and multidisciplinary discussions rather than direct image review by every participant. This distinction is relevant to fetal MRI, where the examination is highly specialized but much downstream reasoning occurs through written interpretation. By deliberately removing image perception from the task, the study isolated a narrower and clinically meaningful question: whether LLMs can reason appropriately from already described fetal MRI findings.

DeepSeek showed a numerically higher principal diagnosis matching rate and higher expert-rated diagnostic-reasoning scores. Its correlation structure reinforces this interpretation: MRI evidence quality, differential diagnostic depth, and logical reasoning were closely linked to diagnostic matching. Thus, when DeepSeek reached the correct diagnosis, it tended to do so through a coherent evidentiary pathway rather than isolated recognition of a diagnostic term. This pattern is compatible with recent observations that reasoning-optimized models, including DeepSeek-R1, can generate structured medical reasoning in benchmarked clinical tasks, although they remain vulnerable to anchoring, incomplete consideration of alternatives, and unnecessarily extended reasoning chains.^[18,19] In the present dataset, however, DeepSeek’s advantage should be interpreted cautiously. Pairwise comparisons for the primary endpoint did not remain statistically significant after correction, indicating a stronger diagnostic core rather than definitive model-level superiority.

Gemini demonstrated the most balanced downstream profile. Although it did not achieve the highest principal diagnosis agreement, it generated the most useful clinical suggestions and the safest communication outputs. From a workflow perspective, this profile may be valuable when the task extends beyond pathology inference to responsible framing of clinical implications for expert review or multidisciplinary discussion.^[20–22] In exploratory correlation analysis, patient-friendliness was negatively associated with communication clarity within Gemini outputs. This finding should be interpreted cautiously. It may reflect a trade-off between empathetic wording and concise clinical communication in some responses.^[23] Such language may appear compassionate but risks reducing precision in safety-sensitive prenatal counseling.

ChatGPT’s results raise a separate issue for evaluating LLM-generated differential diagnoses. In this model, a broader or more elaborate diagnosis list was negatively associated with principal diagnostic matching. This finding cautions against equating diagnostic verbosity with diagnostic quality. In human reasoning, a broad differential diagnosis can indicate expertise when it is prioritized, evidence-based, and linked to discriminating features. In LLM output, however, breadth may also reflect probability dispersion, template completion, or defensive listing when the model fails to converge on a leading explanation. This observation is consistent with prior radiology studies showing that LLMs can generate plausible diagnostic suggestions from text-based imaging descriptions but remain affected by variable accuracy, hallucination, and reliability concerns.^[9,11] It also accords with pediatric radiology evidence that LLM-generated differential diagnoses from text-based image descriptions require structured validation before clinical use.^[24]

The correlation analyses provide information that accuracy metrics alone would obscure. DeepSeek showed compact coupling within the diagnostic core; Gemini showed stronger integration between reasoning and clinical guidance; and ChatGPT showed greater dissociation among response elaboration, reasoning quality, and diagnostic correctness. This internal performance anatomy is informative because it shifts evaluation from a single question – whether the top diagnosis is correct – to a structural question: how different abilities are organized within the same model. This approach is consistent with concerns that medical LLM evaluation often remains fragmented and overly dependent on simplified benchmarks that do not capture clinical utility, safety, workflow fit, or human oversight.^[17,21] For fetal MRI, such multidimensional evaluation is especially important because diagnostic reasoning, management planning, prognosis counseling, and communication safety form a continuous clinical task.

The findings also complicate the assumption that expressed uncertainty is necessarily protective. Across the 3 models, uncertainty disclosure was negatively associated with principal diagnosis matching and reasoning quality. This suggests that uncertainty statements often accompanied weaker analysis rather than calibrated self-monitoring. Recent work has similarly shown that LLM self-reported confidence and uncertainty are unreliable unless supported by external calibration methods, such as sample consistency or other uncertainty proxies.^[25] In prenatal imaging, overconfident incorrect diagnoses can distort clinical prioritization, whereas excessive or poorly structured uncertainty can obscure important findings and increase family anxiety. Future systems should not rely on spontaneous uncertainty statements alone; structured confidence reporting, external calibration, evidence checks, and clinician oversight remain necessary.

Compared with existing radiology and medical LLM literature, this study addresses a narrower but clinically demanding gap. Prior studies have evaluated ChatGPT for differential diagnosis generation from transcribed radiologic findings, LLMs for pediatric radiology case-based diagnosis, LLMs for brain MRI impression generation, and LLMs for structured information extraction from fetal MRI reports.^[11,26,27] These studies established that radiology text contains sufficient diagnostic signal for LLM-based reasoning or information extraction. The present study extends this evidence to fetal CNS MRI cases with postnatal diagnostic confirmation, applying the same prompt architecture across 3 widely used models and evaluating diagnostic matching, reasoning quality, clinical recommendation utility, communication safety, and intra-model metric structure. This design better reflects prenatal imaging practice, where output value depends on whether it can support expert reasoning, multidisciplinary planning, and cautious counseling.

The broader implication is that LLMs should be viewed as role-specific support tools rather than general diagnostic authorities. A reasoning-oriented model may help organize fetal MRI features, propose ranked diagnoses, and identify discriminating findings. A model with stronger workflow or communication performance may be more suitable for drafting management suggestions or preparing multidisciplinary discussion points.^[20,28] Effective deployment would require clear task boundaries, user training, local validation, prospective monitoring, and explicit accountability. Recent radiology guidance further emphasizes that generative AI systems should be assessed not only for statistical performance but also for reproducibility, hallucination risk, privacy, bias, vendor transparency, and human–computer interaction effects.^[29] In fetal MRI, the most defensible near-term role is supervised augmentation: helping specialists structure complex information and identify plausible diagnostic pathways, while leaving final interpretation and counseling to experienced clinicians. These findings are consistent with broader evidence that LLMs may perform well on structured medical reasoning tasks while remaining unsuitable for autonomous clinical use.^[30,31]

Several limitations should be considered. This was a retrospective, single-center study of 85 cases. Although postnatal confirmation strengthened the reference standard, the diagnostic spectrum and reporting language may reflect local referral patterns and institutional practice. The models received standardized text summaries rather than raw MRI images. The text-only design deliberately isolated reasoning from written MRI findings but did not assess direct image perception. Therefore, the present results cannot determine whether multimodal systems can detect subtle fetal neuroanatomical abnormalities, distinguish pathological findings from motion-related artifacts, or integrate information across MRI sequences. The study also did not include comparison with fetal MRI specialists or multidisciplinary human teams; therefore, the results define relative differences among LLMs rather than equivalence to expert performance. Some endpoints, especially recommendation utility and communication safety, depended on expert judgment. Blinded multi-rater assessment and high inter-rater reliability mitigate but do not eliminate this subjectivity. Finally, because commercial LLM platforms are continuously updated, these findings should be interpreted as reflecting the specific model versions and access conditions evaluated during the study period. These constraints limit generalizability but do not alter the main conclusion that fetal MRI LLM performance is multidimensional and model-specific.

Future studies should test these findings in prospective, multicenter cohorts with broader diagnostic distributions, standardized gestational-age strata, and linked genetic and neurodevelopmental outcomes. Direct comparisons among human experts, LLMs, and human-LLM teams will be needed to determine whether model assistance improves diagnostic accuracy, reduces workload, or changes counseling quality. Multimodal research should examine whether image-capable models can integrate fetal MRI sequences with structured clinical text without introducing image-recognition errors or unsupported reasoning. Future work should also align with emerging AI reporting frameworks in medical imaging and clinical decision support, including transparent reporting of model versions, prompting strategies, input data structure, failure modes, and human oversight.^[32,33] Because this study did not include fetal MRI specialists or multidisciplinary human teams as a comparator, the results define relative differences among LLMs rather than equivalence to expert clinical performance. Overall, this study supports LLMs as promising but incomplete tools for fetal MRI text-based reasoning; their clinical value will depend less on isolated diagnostic accuracy than on safe integration into expert-led prenatal imaging workflows.

5. Conclusion

LLMs demonstrated variable and incomplete performance in text-based diagnostic reasoning for fetal CNS MRI. Performance differed across diagnostic agreement, reasoning quality, clinical recommendation utility, and communication safety, indicating that accuracy alone is insufficient for evaluating these systems. Exploratory correlation analyses suggested associations among evaluation metrics but should not be interpreted as evidence of internal model mechanisms or causal relationships. These results support further supervised, prospective evaluation of LLMs as adjunctive text-based decision-support tools, but they do not support autonomous fetal MRI diagnosis, management decision-making, or patient counseling.^[26–28,33]

Author contributions

Methodology: Runze Yu, Simeng Liu.

Investigation: Huiying Li, Tong Su.

Data curation: Hanjie Guan, Yufeng Shen.

Supervision: Yuhui Deng.

Project administration: Deli Zhao.

Writing – original draft: Runze Yu, Miao Peng.

Writing – review & editing: Yuhui Deng, Deli Zhao.