Abstract
Background
The potential for large language models (LLMs) to standardize clinical decision-making in ARF, particularly in optimizing high-flow nasal cannula (HFNC) therapy application, is gaining interest due to the condition’s complex etiology. This study aimed to evaluate the clinical utility of artificial intelligence (AI) models in aligning with the European Respiratory Society (ERS) clinical practice guidelines for the use of HFNC in acute respiratory failure (ARF).
Methods
Eight advanced LLMs were assessed across eight representative clinical scenarios related to HFNC use in ARF. Each model’s responses were independently scored by three independent reviewers across four domains: accuracy, overconclusiveness, supplementary value, and completeness, using a 5-point Likert scale. Inter-rater reliability was measured by Fleiss’ Kappa. Readability metrics, including Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL), were also analyzed.
Results
No differences were detected in accuracy or overconclusiveness (both P>0.99). Significant differences emerged in supplementary value among models (P<0.0001). DeepSeek-V3.1 ranked highest, significantly outperforming Claude and ChatGPT models (P<0.05). For completeness, ChatGPT-4o scored lower than both DeepSeek-V3.1 (P=0.006) and DeepSeek-R1 (P=0.01). Readability analyses revealed that DeepSeek-V3.1 achieved the highest FRE score, while DeepSeek-R1 had the lowest FKGL, reflecting better textual accessibility. Inter-rater reliability was substantial (κ=0.781).
Conclusions
Evaluations of LLMs for HFNC guideline interpretation show that while all are accurate, DeepSeek-V3.1 and R1 excel in completeness, supplemental detail, and readability, marking them as more suitable for ARF clinical decision support. Their integration into critical workflows still demands expert supervision.
Keywords: Acute respiratory failure (ARF), high-flow nasal cannula (HFNC), artificial intelligence (AI), large language models (LLMs)
Highlight box.
Key findings
• This study evaluated eight advanced large language models (LLMs) against European Respiratory Society (ERS) guidelines for High-Flow Nasal Cannula (HFNC) therapy in Acute Respiratory Failure (ARF). All models achieved high accuracy without overconclusiveness. However, significant disparities emerged in Supplementary Value and Completeness, with DeepSeek-V3.1 and R1 outperforming proprietary counterparts.
What is known and what is new?
• DeepSeek variants generated the most linguistically accessible content for time-sensitive settings, whereas other models produced outputs requiring higher literacy levels. Inter-rater reliability was substantial (κ=0.781).
• We identified that LLMs are optimized for safety rather than adaptive reasoning, limiting their utility in complex cases involving multimorbidity or conflicting evidence.
What is the implication, and what should change now?
• While LLMs show promise as adjuncts, developers must prioritize adaptive reasoning and completeness over mere accuracy. Future validation should focus on dynamic clinical settings and impacts on decision-making speed and patient safety.
Introduction
Acute respiratory failure (ARF) is a global healthcare challenge associated with high morbidity, mortality, and intensive care resource consumption (1-3). While conventional oxygen therapy (COT) and noninvasive ventilation (NIV) are established treatments, high-flow nasal cannula (HFNC) has emerged as a key alternative, providing physiological benefits such as improved comfort, humidified gas delivery, alveolar recruitment, and reduced work of breathing (4,5). These advantages may help prevent lung function decline and lower intubation needs in specific ARF subgroups (6,7). The optimal choice of noninvasive respiratory support for ARF remains uncertain due to the heterogeneity of ARF etiologies, such as hypoxemic versus hypercapnic respiratory failure, postoperative or post-extubation ARF, and coronavirus disease 2019 (COVID-19) pneumonia. To address these issues, the European Respiratory Society (ERS) issued evidence-based guidelines to standardize HFNC use in adults with ARF (8).
Recent advances in artificial intelligence (AI), specifically large language models (LLMs) such as ChatGPT-5 and DeepSeek-R1, represent transformative tools for clinical decision support (9,10). These transformer-based models utilize probabilistic “Chain-of-Thought” (CoT) reasoning to decompose complex medical queries into stepwise clinical logic, simulating clinician cognition in diagnostic and therapeutic planning (11). Despite their potential to streamline guideline adherence, concerns persist regarding the reliability of LLMs outputs due to limited validation against rigorous clinical standards (12). Although studies have evaluated the performance of LLMs in specialties such as digestive diseases and orthopedic pathology (12-15), their application to complex ARF scenarios remains unexplored. In ARF, evidence-based selection of respiratory support is critical, as it directly influences outcomes such as intubation timing and mortality.
To address this gap, we benchmarked major LLMs against ERS task force recommendations for HFNC use in ARF. Using standardized clinical vignettes across diverse ARF subtypes, we quantified the models’ alignment with evidence-based indications, contraindications, and escalation criteria. This work establishes an evaluation framework for LLMs clinical reasoning, guiding the safe integration of AI into respiratory care to enhance clinician decision support and ensure guideline fidelity.
Methods
Study design
This research employed publicly accessible models such as ChatGPT, DeepSeek, Grok, and Claude. As the research did not involve human subjects, institutional review board (IRB) approval was not required, consistent with ethical research guidelines. All data were anonymized and sourced from publicly accessible datasets, with no patient-level information included.
Evaluation process
To ensure the impartiality and reliability of the responses generated by these models, a rigorous methodology was employed. Each question was submitted independently to the respective versions of LLMs without any prior prompting or contextual carryover from previous interactions. To further enhance the integrity of the results, a fresh chat session was initiated for every individual question, thereby minimizing any potential residual influence from prior queries.
The assessment group consisted of three reviewers and one adjudicator. All held advanced clinical degrees (Master of Medicine degree or equivalent) and had more than 15 years’ experience in respiratory medicine or critical care. Reviewers were chosen for knowledge of ARF and familiarity with the interpretation of evidence-based guidelines. Prior to evaluation, all reviewers underwent a standard training session where they used a pilot set of responses to adjust the score criteria so that the 5-point Likert scale would be consistently applied across the four assessment domains: accuracy, overconclusiveness, supplementary value, and completeness. The three main reviewers individually evaluated every model’s answers without knowing what scores other people gave or which LLM it belonged to. In the event of disagreements about scoring, we asked the fourth reviewer (adjudicator) to help us decide.
This methodological rigor was implemented to uphold the objectivity and reproducibility of the findings. Furthermore, these models are openly available and have shown significance in contemporary medical literature, indicating their potential to enhance clinical workflows (16-19).
Guideline-based evaluation questions
The evaluation revolved around the eight basic clinical questions from ERS guidelines for using HFNC in ARF. These questions were selected to include main decision points during real-life application of the guideline, such as indication, contraindication, monitoring and escalation. The whole list is provided below (8).
Should HFNC or COT be used in patients with acute hypoxaemic respiratory failure?
Should HFNC or NIV be used in patients with acute hypoxaemic respiratory failure?
Should HFNC or COT be used during breaks from NIV in patients with acute hypoxaemic respiratory failure?
Should HFNC or COT be used in post-operative patients after extubation?
Should HFNC or NIV be used in post-operative patients after extubation?
Should HFNC or COT be used in nonsurgical patients after extubation?
Should HFNC or NIV be used in nonsurgical patients after extubation?
Should HFNC or NIV be used in patients with acute hypercapnic respiratory failure?
Model selection
The evaluated models included: ChatGPT-5, ChatGPT-4o, DeepSeek-V3.1, DeepSeek-R1, Grok-4, Grok-3, Claude Opus 4.1, and Claude-3.7.
This study selected a diverse set of advanced general-purpose LLMs based on their architectural significance, accessibility, and relevance to clinical reasoning tasks. Specifically, ChatGPT was included as a leading proprietary model known for robust general-purpose reasoning, serving as a benchmark for performance comparison. DeepSeek, a state-of-the-art open-source model optimized for logical reasoning and mathematical tasks, was chosen for its open-weight design, enabling local deployment and addressing data privacy needs in healthcare. Grok, a high-performance proprietary model, was selected for its strength in complex reasoning and specialized knowledge, ideal for evaluating advanced clinical decision-making. Claude, another top-tier proprietary model, was included for its nuanced language understanding and suitability for safety-critical healthcare applications (16,17).
The selection included both open-source and proprietary LLMs to provide a comprehensive analysis of the current LLMs landscape. These models represent state-of-the-art general-purpose LLMs widely utilized in research, including medical applications.
Assessment criteria
A 5-point Likert scale was used to assess four dimensions of each model’s response quality: accuracy, overconclusiveness, supplementary value, and completeness (13,20). Before formal evaluation, the scoring criteria underwent calibration among assessors and pilot testing.
❖ Accuracy: 1, completely incorrect; 2, more incorrect than correct (>75% incorrect); 3, approximately equal correct and incorrect; 4, more correct than incorrect (>75% correct); 5, completely correct.
❖ Overconclusiveness: 1, non-overconclusive (0% conflicting); 2, minimally overconclusive (<25% conflicting); 3, partially overconclusive (50% conflicting); 4, mostly overconclusive (>75% conflicting); 5, fully overconclusive (100% conflicting).
❖ Supplementary value: 1, no supplementary value (0% added); 2, low supplementary value (25% added); 3, moderate supplementary value (50% added); 4, high supplementary value (>75% added); 5, exceptional supplementary value (100% novel).
❖ Completeness: 1, very incomplete (0–25%); 2, incomplete (25–50%); 3, moderate (50–75%); 4, complete (>75%); 5, very complete (100%).
In this study, we calculated the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) for each model’s responses (20). A higher FRE score indicates easier readability, while a lower FKGL suggests the text is suitable for readers at a lower grade level.
Statistical analysis
SPSS 26.0 was used for statistical analysis. One-way analysis of variance (ANOVA) with post-hoc Tukey tests was applied to compare performance differences across models under identical dimensions. Fleiss’ kappa, serving as a generalization of this statistic, was used in SPSS to evaluate the consistency among the three raters for each model’s response qualities. P<0.05 was considered to have statistical significance.
Results
In this study, we evaluated the performance of eight LLMs. There were a total of 8 clinical scenarios that were included in the ERS clinical practice guidelines. The outputs were evaluated accordingly on the Likert scale for each dimension. This approach allowed for a comprehensive comparison of each model’s alignment with established clinical guidelines. The responses generated by all evaluated models were systematically documented in, available online https://cdn.amegroups.cn/static/public/jtd-2026-1-0230-1.docx.
Performance on Likert dimensions
No significant differences were observed among the eight models in terms of Accuracy (P>0.99), Overconclusiveness (P>0.99). Significant differences in Supplementary Value were observed among the eight models (F7, 56=7.910, P<0.0001, η²=0.4972). The performance ranking was as follows: DeepSeek-V3.1 (3.000±0.504) > DeepSeek-R1 (2.793±0.503) > Grok-3 (2.375±0.415) > Grok-4 (2.374±0.331) > Claude-3.7 (2.250±0.611) > Claude Opus 4.1 (2.206±0.502) > ChatGPT-5 (1.918±0.153) > ChatGPT-4o (1.668±0.253). Post hoc Tukey’s HSD tests revealed DeepSeek-V3.1 had significantly higher supplementarity than Claude-3.7 (P=0.02), Claude Opus 4.1 (P=0.01), ChatGPT-5 (P=0.002), and ChatGPT-4o (P<0.0001). DeepSeek-R1 had significantly higher supplementarity than ChatGPT-5 (P=0.003) and ChatGPT-4o (P<0.0001). ChatGPT-4o had significantly lower supplementarity than Grok-3 and Grok-4 (both P<0.05).
Among the eight AI models, there were significant differences in terms of completeness (F7, 56=1.799, P=0.005, η²=0.2963). Post hoc Tukey’s HSD tests revealed ChatGPT-4o (4.291±0.485) had significantly lower completeness than DeepSeek-V3.1 (4.959±0.117, P=0.006) and DeepSeek-R1 (4.918±0.153, P=0.01). No other pairwise comparisons reached statistical significance (adjusted P>0.05 for all other model combinations) (Figure 1).
Figure 1.
Comparison of (A) accuracy, (B) overconclusiveness, (C) supplementary value and (D) completeness value scores across models. *, P<0.05; **, P<0.01; ***, P<0.001; ****, P<0.0001; ns, not significant.
Readability metrics
Significant differences in FRE scores were detected (F7, 56=9.371, P<0.0001, η²=0.5395). Claude Opus 4.1 (16.705±6.173) exhibited significantly lower readability than DeepSeek-V3.1 (36.951±5.343, P<0.0001), DeepSeek-R1 (33.449±7.551, P<0.0001), Grok-4 (31.948±0.331, P<0.0001), Grok-3 (28.375±4.405, P=0.003), ChatGPT-4o (26.990±5.011, P=0.01), ChatGPT-5 (26.963±0.485, P=0.01) and Claude-3.7 (25.673±4.068, P=0.045). DeepSeek-V3.1 exhibited significantly higher readability than ChatGPT-4o (P=0.02), ChatGPT-5 (P=0.01), and Claude-3.7 (P=0.004, Figure 2).
Figure 2.
Comparison of FRE scores across models. *, P<0.05; **, P<0.01; ****, P<0.0001. FRE, Flesch Reading Ease.
Significant differences in FKGL were observed among the eight models (F7, 56=12.88, P<0.0001, η²=0.6169). The performance ranking was as follows: Claude Opus 4.1 (14.278±1.563) > ChatGPT-4o (13.603±1.019) > Claude-3.7 (13.174±1.173) > ChatGPT-5 (13.013±1.531) > Grok-3 (11.740±0.783) > Grok-4 (11.018±0.806) > DeepSeek-V3.1 (10.834±0.700) > DeepSeek-R1 (10.423±1.156). Post hoc tests revealed DeepSeek-R1 was significantly easier to understand than Claude Opus 4.1 (P<0.0001), ChatGPT-4o (P<0.0001), Claude-3.7 (P=0.003) and ChatGPT-5 (P=0.0007). DeepSeek-V3.1 was significantly easier to understand than Claude Opus 4.1 (P<0.0001), ChatGPT-4o (P=0.0002), Claude-3.7 (P=0.0003) and ChatGPT-5 (P=0.007). Grok-4 was significantly easier to understand than Claude Opus 4.1 (P<0.0001), ChatGPT-4o (P=0.0007), Claude-3.7 (P=0.008) and ChatGPT-5 (P=0.02). Grok-3 was significantly easier to understand than Claude Opus 4.1 (P=0.0009) and ChatGPT-4o (P=0.03). No other pairwise comparisons reached statistical significance (adjusted P>0.05 for all other model combinations, Figure 3).
Figure 3.
Comparison of FKGL across models. *, P<0.05; **, P<0.01; ***, P<0.001; ****, P<0.0001. FKGL, Flesch-Kincaid Grade Level.
Inter-rater reliability
The inter-rater reliability, assessed using Fleiss’ Kappa, was found to be 0.781 (P<0.0001) according to Landis and Koch’s criteria, this kappa value demonstrates substantial agreement among the raters, suggesting that the observed consistency was not attributable to random chance.
Discussion
As AI tools gain traction in delivering clinical information, it is essential to assess their reliability and alignment with such guidelines. In this study, we evaluated the performance of eight LLMs against the ERS clinical practice guidelines (8). The evaluation incorporated eight clinically relevant scenarios, reflecting diverse situations encountered in clinical practice. Evaluations of LLMs for HFNC guideline interpretation show that while all are accurate, with no statistically significant differences in performance, and none of the models exhibited overconclusive tendencies. However, DeepSeek-V3.1 and R1 excel in completeness, supplemental detail, and readability, marking them as more suitable for ARF clinical decision support.
Homogeneity in guideline adherence
They are all so accurate, and none of them is too conclusive. This demonstrates contemporary LLM’s basic capacity to accurately replicate well-established international standards, possibly originating from their common training data as well as similar transformer-based models.
Clinically safe as well, it would be reassuring if the results weren’t conclusive. High-stakes situations call for the use of AI systems that should refrain from asserting themselves too strongly when there is a lack of certainty about facts, so that wrong clinical decisions don’t get taken. But although this indicates strict adherence to procedure, there’s also a problem. They might have been optimized more toward playing it safe than adapting and thinking critically. However, these findings shouldn’t be considered reliable under all clinical conditions, especially in cases when several morbidities are present or conflicting guidelines happen to come up. In terms of future research, models must also be tested in contexts where nuanced clinical judgment is required rather than simply repeating guidelines.
Divergence in supplementary value and completeness
Notable and statistically significant differences emerged among the eight AI models in both supplementary value and completeness. For supplementary value, DeepSeek-V3.1 demonstrated superior supplementarity, significantly outperforming the Claude series and ChatGPT series. DeepSeek-R1 also showed significantly higher supplementarity than ChatGPT series. DeepSeek-V3.1 and DeepSeek-R1 outperformed ChatGPT-4o in completeness.
The difference points to a big gap between simply quoting guidelines and making them rich with contextual knowledge. The models which had more added up, liked using pathophysiological reasoning or recently found evidence that is not yet part of the written-down guideline. But it takes careful reading, because enrichment helps you make better decisions, but it can add speculative detail away from the core protocols based on facts. Similarly, variability in completeness also causes problems for clinical safety; if key monitoring parameters or escalation steps are missed, it can mean poor care in emergency cases. Findings showed that performance of clinical LLM depends upon the model used and developers need to focus on providing complete, guideline-based information reliably.
Clinical utility and readability
We found that model outputs vary widely in how easy they are to read. In fast-paced clinical settings, such as emergency or intensive care units, readability is critical. Clear recommendations help clinicians understand and act quickly. This reduces their cognitive load. On the other hand, text that requires advanced literacy can slow comprehension. It may increase the risk of misunderstanding under pressure, limiting its practical value.
Although we used FRE and FKGL to evaluate accessibility, we should recognize a key trade-off. Simplicity can conflict with clinical precision. High FRE scores indicate that text is easier to read. However, easy reading does not always mean better clinical usefulness. Overly simple language may leave out important technical details. It may also omit specific pharmacological information. In addition, it can fail to capture the nuanced reasoning required for complex cases. Therefore, the most useful outputs should achieve a balance. The text should be clear and easy to understand. At the same time, it should remain detailed and accurate. Future model development should focus on this balance. Developers should ensure that simplification improves understanding. They should also make sure that clinical accuracy is not compromised.
Evaluating AI-generated medical content goes beyond accuracy. We must also consider how the information is presented. Moving forward, optimizing readability in model fine-tuning will be crucial. Equally important is improving the completeness of the content. A dual focus on clarity and precision, along with comprehensiveness, will be essential. This approach will help build trust and support the safer integration of AI into clinical workflows.
Limitations
Our study has limitations. First, in complex clinical decision-making, there is rarely a single, universally correct answer. Consequently, even expert reviewers cannot definitively determine the “correct” output of an AI model. Our evaluation framework was therefore not designed to validate clinical outcomes against a singular gold standard. Instead, model performance was assessed based on adherence to the ERS clinical practice guidelines, focusing on guideline fidelity rather than absolute clinical truth.
Second, our study was limited to eight HFNC-related clinical scenarios, and the results may not generalize to other clinical contexts or specialties. Additionally, while Likert-scale ratings provide structured evaluation, they remain subject to human interpretation. Future research should expand the range of clinical scenarios, incorporate objective performance metrics, and explore user-centered outcomes such as decision-making efficiency and patient safety.
Lastly, while all eight HFNC-related scenarios were distinct, they were relatively narrow and lacked dangerous situations or escalating decisions that require complex judgment beyond strict guideline adherence. Furthermore, although guideline-based, our evaluation relied heavily on static indices like the respiratory rate oxygenation (ROX) index, which is not a universal gold standard and may vary across populations; future studies should consider dynamic values and newer indices. Future research should incorporate multifactorial instances involving a high level of contextual complexity and utilize objective performance metrics, as well as explore user-focused outcomes like decision-making effectiveness and patient safety, thereby verifying the LLMs’ true utility in clinical decision-making.
Conclusions
This study systematically evaluated eight LLMs against the ERS guidelines for HFNC therapy in ARF, revealing uniform high accuracy and absence of overconclusiveness across all models, indicating strong alignment with evidence-based recommendations. However, significant model-dependent disparities were observed in supplementary value and completeness, with DeepSeek-V3.1 and DeepSeek-R1 outperforming others in providing contextual insights and fully developed recommendations, while ChatGPT-4o underperformed in these domains. Pronounced differences in readability were also noted, with DeepSeek variants generating more linguistically accessible content suitable for time-sensitive clinical environments, whereas models like Claude Opus 4.1 produced outputs requiring higher educational levels for comprehension.
The findings underscore that while LLMs demonstrate potential as adjunct tools for clinical decision support, their practical utility varies significantly based on comprehensiveness, contextual relevance, and linguistic clarity. Future efforts should focus on standardized evaluation frameworks and model optimization to enhance both reliability and usability across diverse clinical scenarios, ensuring AI-generated recommendations are not only accurate but also actionable and accessible in real-world healthcare settings.
Supplementary
The article’s supplementary files as
Acknowledgments
We used AI tools ChatGPT-5, ChatGPT-4o, DeepSeek-V3.1, DeepSeek-R1, Grok-4, Grok-3, Claude Opus 4.1, and Claude-3.7 to answer questions from the research survey, and the full answers are attached to the supplementary materials.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Footnotes
Funding: This research was supported by the Medical and Health Science and Technology Project of Zhejiang Provincial (No. 2025KY1168), National Natural Science Foundation of China (Nos. 82402582 and 82202561), Zhejiang Provincial Natural Science Foundation of China (No. LQ24H150003), Hangzhou Joint Fund of Zhejiang Provincial Natural Science Foundation of China (No. LHZQN26H290002), Traditional Chinese Medicine Science and Technology Project of Zhejiang Province (No. 2024ZL716), and The Construction Fund of Key Medical Disciplines of Hangzhou (Nos. 2025HZZD04, 2025HZPY06, and 0020200484).
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2026-1-0230/coif). Hongxia Wu reports funding from Zhejiang Provincial Health Commission (The Medical and Health Science and Technology Project of Zhejiang Provincial, Grant: 2025KY1168). S.W. reports funding from Zhejiang Provincial Department of Science and Technology (Hangzhou Joint Fund of Zhejiang Provincial Natural Science Foundation of China) (No. LHZQN26H290002). Y.L. reports funding from Zhejiang Provincial Administration of Traditional Chinese Medicine (Traditional Chinese Medicine Science and Technology Project of Zhejiang Province) (No. 2024ZL716). Y.Y. reports funding from National Natural Science Foundation of China (No. 82202561). H.Z. reports funding from Hangzhou Municipal Health Commission (The Construction Fund of Key Medical Disciplines of Hangzhou) (No. 2025HZPY06). Hanbin Wang reports funding from National Natural Science Foundation of China (No. 82402582), Zhejiang Provincial Department of Science and Technology (Zhejiang Provincial Natural Science Foundation of China) (No. LQ24H150003), and Hangzhou Municipal Health Commission (The Construction Fund of Key Medical Disciplines of Hangzhou) (Nos. 2025HZZD04 and 0020200484). The other authors have no conflicts of interest to declare.
References
- 1.Gong MN, Schenk L, Gajic O, et al. Early intervention of patients at risk for acute respiratory failure and prolonged mechanical ventilation with a checklist aimed at the prevention of organ failure: protocol for a pragmatic stepped-wedged cluster trial of PROOFCheck. BMJ Open 2016;6:e011347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Custodero C, Gandolfo F, Cella A, et al. Multidimensional prognostic index (MPI) predicts non-invasive ventilation failure in older adults with acute respiratory failure. Arch Gerontol Geriatr 2021;94:104327. [DOI] [PubMed] [Google Scholar]
- 3.Chen L, Rackley CR. Diagnosis and Epidemiology of Acute Respiratory Failure. Crit Care Clin 2024;40:221-33. [DOI] [PubMed] [Google Scholar]
- 4.Ricard JD, Roca O, Lemiale V, et al. Use of nasal high flow oxygen during acute respiratory failure. Intensive Care Med 2020;46:2238-47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kang BJ, Koh Y, Lim CM, et al. Failure of high-flow nasal cannula therapy may delay intubation and increase mortality. Intensive Care Med 2015;41:623-32. [DOI] [PubMed] [Google Scholar]
- 6.Renda T, Corrado A, Iskandar G, et al. High-flow nasal oxygen therapy in intensive care and anaesthesia. Br J Anaesth 2018;120:18-27. [DOI] [PubMed] [Google Scholar]
- 7.Guyatt GH, Oxman AD, Santesso N, et al. GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes. J Clin Epidemiol 2013;66:158-72. [DOI] [PubMed] [Google Scholar]
- 8.Oczkowski S, Ergan B, Bos L, et al. ERS clinical practice guidelines: high-flow nasal cannula in acute respiratory failure. Eur Respir J 2022;59:2101574. [DOI] [PubMed] [Google Scholar]
- 9.Singhal K, Azizi S, Tu T, et al. Large language models encode clinical knowledge. Nature 2023;620:172-80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Liévin V, Hother CE, Motzfeldt AG, et al. Can large language models reason about medical questions? Patterns (N Y) 2024;5:100943. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Shah HA, Househ M. Chain of Thought Strategy for Smaller LLMs for Medical Reasoning. Stud Health Technol Inform 2025;327:783-7. [DOI] [PubMed] [Google Scholar]
- 12.Giuffrè M, Kresevic S, You K, et al. Systematic review: The use of large language models as medical chatbots in digestive diseases. Aliment Pharmacol Ther 2024;60:144-66. [DOI] [PubMed] [Google Scholar]
- 13.Mejia MR, Arroyave JS, Saturno M, et al. Use of ChatGPT for Determining Clinical and Surgical Treatment of Lumbar Disc Herniation With Radiculopathy: A North American Spine Society Guideline Comparison. Neurospine 2024;21:149-58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dong C, Qiu X, Deng J, et al. Comparative evaluation of large language models in delivering guideline-compliant recommendations for topical NSAID use in musculoskeletal pain: a multidimensional analysis. Clin Rheumatol 2025;44:4703-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Deng J, Qiu X, Dong C, et al. Evaluating ChatGPT and DeepSeek in postdural puncture headache management: a comparative study with international consensus guidelines. BMC Neurol 2025;25:264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Chen YQ, Yu T, Song ZQ, et al. Application of Large Language Models in Drug-Induced Osteotoxicity Prediction. J Chem Inf Model 2025;65:3370-9. [DOI] [PubMed] [Google Scholar]
- 17.Duey AH, Nietsch KS, Zaidat B, et al. Thromboembolic prophylaxis in spine surgery: an analysis of ChatGPT recommendations. Spine J 2023;23:1684-91. [DOI] [PubMed] [Google Scholar]
- 18.Sandmann S, Hegselmann S, Fujarski M, et al. Benchmark evaluation of DeepSeek large language models in clinical decision-making. Nat Med 2025;31:2546-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wu H, Deng J, Qiu X, et al. Comparative assessment of large language models in diabetic foot infection management: alignment with IWGDF/IDSA guidelines. Front Endocrinol (Lausanne) 2026;17:1667159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Wang S, Wang Y, Jiang L, et al. Assessing the clinical support capabilities of ChatGPT 4o and ChatGPT 4o mini in managing lumbar disc herniation. Eur J Med Res 2025;30:45. [DOI] [PMC free article] [PubMed] [Google Scholar]