Evaluating the Performance and Fragility of Large Language Models on the Self-Assessment for Neurological Surgeons

Abstract

BACKGROUND AND OBJECTIVES:

The Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons questions are widely used by neurosurgical residents to prepare for written board examinations. Recently, these questions have also served as benchmarks for evaluating large language models' (LLMs) neurosurgical knowledge. LLMs show significant promise for transforming neurosurgical practice; however, they are susceptible to in-text distractions and confounding factors. Given the increasing use of generative artificial intelligence and ambient dictation technologies, clinical text is at a larger risk for the inclusion of extraneous details. The aim of this study was to assess the performance of state-of-the-art LLMs on neurosurgery board–like questions and to evaluate their robustness to the inclusion of distractor statements.

METHODS:

A comprehensive evaluation was conducted using 28 state-of-the-art LLMs. These models were tested on 2904 neurosurgery board examination questions derived from the Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons. In addition, the study introduced a distraction framework to assess the fragility of these models. The framework incorporated simple, irrelevant distractor statements containing polysemous words with clinical meanings used in nonclinical contexts to determine the extent to which such distractions degrade model performance on standard medical benchmarks.

RESULTS:

Six of the 28 tested LLMs achieved board-passing outcomes, with the top-performing models scoring over 15.7% above the passing threshold. When exposed to distractions, accuracy across various model architectures was significantly reduced—by as much as 20.4%—with 1 model failing that had previously passed. Both general-purpose and medical open-source models experienced greater performance declines compared with proprietary variants when subjected to the added distractors.

CONCLUSION:

While current LLMs demonstrate an impressive ability to answer neurosurgery board–like examination questions, their performance is markedly vulnerable to extraneous, distracting information. These findings underscore the critical need for developing novel mitigation strategies aimed at bolstering LLM resilience against in-text distractions, particularly for safe and effective clinical deployment.

ABBREVIATIONS:

AI

artificial intelligence

CNS-SANS

The Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons

LLMs

large language models.

Large language models (LLMs) are rapidly entering clinical neurosurgery, with growing applications in imaging interpretation, intraoperative decision making, documentation workflows, and education.^1-5 The Congress of Neurological Surgeons' Self-Assessment in Neurological Surgery (CNS-SANS) question bank—long regarded as a gold standard for both resident training and formal assessment—has recently emerged as a benchmark for evaluating whether LLMs can perform at or above the level of a neurosurgical resident in clinical knowledge.^6-11 Recent studies show that leading LLMs such as GPT-3.5 and GPT-4 can exceed board-passing thresholds on neurosurgery^6,7 and general medicine (United States Medical Licensing Exam) boards–style questions.^12,13 Furthermore, even open-source, light-weight language models that can run on a phone, such as MedMobile, perform well across multiple medical question and answer benchmarks.¹⁴ Collectively, these findings suggest that LLMs may soon be viable tools for clinical education and point-of-care decision support in neurosurgery.

However, these benchmarks typically involve clean, highly structured prompts that lack the ambiguity and noise of real-world clinical documentation—where tangential information, inconsistently phrased findings, and incomplete data are common. In practice, neurosurgeons must filter noise to identify critical facts, such as a subtle neurological sign or a relevant imaging finding, that inform life-altering decisions. Unlike human clinicians, who are trained to filter irrelevant content, LLMs appear highly susceptible to noisy or distracting inputs. Omar et al¹⁵ demonstrated that embedding fabricated clinical facts into prompts led to confident hallucinations in over 50% to 80% of cases. Vishwanath et al¹⁶ found that inserting irrelevant distractors, such as polysemous terms used in nonclinical contexts, into clinical multiple choice questions decreased accuracy by up to 18%, and that fine-tuning or retrieval augmentation failed to mitigate this effect. Ness et al¹⁷ confirmed that even subtle changes to medical prompts can flip correct answers to incorrect ones—mistakes that trained clinicians would not make.

Despite growing enthusiasm for LLMs in neurosurgery, their ability to deal with noisy or adversarial input remains poorly understood. This is especially pertinent as real-world clinical scenarios are replete with extraneous details—patient histories often include tangential information, and not every detail in a case presentation is pertinent to the diagnosis or management question at hand. These vulnerabilities are especially concerning amid the rapid deployment of ambient artificial intelligence (AI) documentation tools in health care. Systems such as AI scribes transcribe patient encounters and draft clinical notes in real time, promising to reduce clinician administrative burden and streamline workflows.^18-20 However, they also introduce extraneous or poorly contextualized information into the clinical record—raising the risk that downstream LLMs will misinterpret or overemphasize nonessential or misleading content. In high-stakes fields such as neurosurgery, where precision is paramount, an LLM misled by noise could produce unsafe recommendations with direct implications for patient safety.

To address these gaps, we make 2 primary contributions. First, we evaluate 28 LLMs—spanning open-source, proprietary, general-purpose, and medically fine-tuned models—on 2904 CNS-SANS board–style questions across all neurosurgical subspecialties. Second, we evaluate model resilience to distractions by systematically injecting clinically irrelevant distractors into prompts. This allows us to simulate the complexity of real-world documentation and assess how such noise degrades model performance. Our goal is to benchmark accuracy and identify the conditions under which LLMs fail—and, ultimately, to elucidate the clinical risks of those failures as neurosurgery moves toward AI integration.

METHODS

Preapprovals and Ethics

The Congress of Neurological Surgeons approved the use of SANS questions in this study. No Institutional Review Board approval was sought as this study did not involve human subjects. No procedures were performed as a part of this artificial intelligence model benchmarking study.

SANS Benchmark

Our primary benchmark was derived from the SANS questions. Developed by the CNS, these questions are a standard resource for neurosurgery residents preparing for the examinations conducted by the American Board of Neurological Surgeons. The CNS provided a total of 3965 questions for our evaluation. Of these, 1061 questions included supplementary assets, such as images, and were excluded, leaving 2904 text-only questions as our benchmark. Evaluation on the CNS-SANS test bank was conducted with explicit permission from the CNS. A model evaluation is considered to have passed the CNS-SANS if it surpasses a score of 72%.

SANS With Distraction Benchmark Curation

We began by isolating clinically relevant terms from the first incorrect answer options of each SANS item. By selecting only distractor terms, rather than those associated with the correct response, we craft content lacking clinical utility. Specifically, we parsed each question's list of wrong choices and extracted medical concepts (eg, diseases, conditions, or procedures). For example, if “cancer” appeared among the distractors for a given question, we designated that phrase for possible inclusion in a nonclinical context. This approach exploits the natural variety of erroneous options, often featuring commonplace medical terminology amenable to reinterpretation.

Next, we used GPT-4o to convert each extracted term into a concise, coherent sentence that uses the medical word in a socially or culturally oriented—but clinically irrelevant—context. Prompts instructed the model to generate statements that read fluidly yet convey no diagnostic or therapeutic information. For instance, we might add in the sentence “The patient's zodiac sign is Cancer,” in which “Cancer” refers to the astrological sign rather than the malignancy. These confounding sentences were then interwoven into the original SANS questions, yielding augmented items with noise (see Supplemental Digital Content 1, http://links.lww.com/NEU/F139, for example, augmented prompt and model response). By systematically introducing irrelevant medical language, we assessed the impact of such distractions on the model's ability to select the correct clinical answer. The overall workflow is illustrated in Figure 1.

FIGURE 1.

Schematic of our study. Problems from the SANS were extracted and fed into LLMs and evaluated for their accuracy in answering the questions correctly (black-orange arrows). The same extracted problems were also used to extract an incorrect answer choice, extract a clinically relevant term from that incorrect answer choice using GPT-4o, and algorithmically embed the clinically relevant term from the wrong answer choice into the original question. The modified problems, including the questions with “distractors,” were then similarly fed into the LLMs and evaluated for accuracy (black-blue arrows). CNS-SANS, The Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons; LLMs, large language models.

Model Evaluation

Accuracy on the SANS and SANS-Noisy benchmarks was assessed by applying exact string matching to each model's chain-of-thought output. All models were run with deterministic decoding—that is temperature set to zero—and without ensembling, unless the model inherently disallows temperature control. We used the providers' default inference parameters in such cases (eg, the OpenAI reasoning models). All open-source and medically fine-tuned models were served using the PyTorch package,²¹ the Hugging Face Transformers library,²² and the vLLM inference library²³ on identical A100 hardware to ensure a fair comparison. Inference from proprietary systems was obtained through their official Application Programming Interfaces, with each call executed according to the vendor's recommended usage guidelines.

Prompting

Prompting templates and prompt engineering for model evaluations follows the method detailed in the study by Vishwanath et al.¹⁵

Categorization of CNS-SANS Questions

For each question in the CNS-SANS test bank, we use a prompting scheme (Supplemental Digital Content 2, http://links.lww.com/NEU/F140) to identify its theme from 12 official categories of the neurosurgery board examination: Peripheral Nerve, Functional, Fundamentals, Spine, Pain, Tumor, Pediatrics, Trauma, Vascular, Neuropathology, Neuroradiology, and Other General.

Statistical Analysis

All statistical analysis was performed in GraphPad Prism 10.4.2 (Dotmatics) with significance determined at P < .05. Analysis of variance with Tukey post hoc analysis was conducted to reveal any significant differences between LLM accuracies. The Spearman rank correlation coefficient was calculated and tested for any significant associations between rankings. All results are reported as the mean ± SE of the mean unless otherwise specified. To develop intramodel statistical analysis, we perform the McNemar test between model evaluations on the baseline and distracted benchmarks.

Data Availability

Owing to the proprietary nature of the SANS data set, the data sets generated or analyzed during this study will NOT be released.

RESULTS

Distractors' Impact on SANS Accuracy

Once we input the unmodified questions and the questions with distractors into the LLMs, the accuracies of each model were evaluated and subsequently analyzed (Figure 1). Regardless of the model, the presence of distractors in the questions led to a decrease in the accuracy of the LLMs (Figure 2A). However, we observed that the reductions were not symmetrical across the different model classifications of general open-source; medical open-source; or proprietary.

FIGURE 2.

Accuracy of LLMs in answering SANS questions with and without distractors present by model classification. A, Accuracy of LLMs in answering SANS questions with distractors (green) and the respective loss from the baseline accuracy without any distractors (red). The baseline accuracy of a given LLM is the respective total length of the bar (green plus red). Models are grouped by their classifications as general open-source (orange), medical open-source (blue), and proprietary (black). B, Baseline accuracy of general open-source, medical open-source, and proprietary LLMs in predicting answers to SANS questions without any distractors. C, Accuracy of general open-source, medical open-source, and proprietary LLMs in predicting answers to SANS questions with distractors present. D, Loss in accuracy of general open-source, medical open-source, and proprietary LLMs from the baseline accuracy to the accuracy with distractors present. Bars indicate significant differences (P < .05 one-way analysis of variance with Tukey post hoc analysis, *P < .05, ***P < .001, ****P < .0001; general open-source, n = 19; medical open-source, n = 3; proprietary, n = 6). LLMs, large language models; SANS, self-assessment in neurological surgery.

Considering just the baseline performance, the proprietary models significantly outperformed both open-source models (P = .0280 compared with medical open-source; P < 10⁻⁴ compared with general open-source; Figure 2B). Interestingly, the same trend held for the performance with distractors present within the questions (P = .0188 comparing proprietary with medical open-source; P < 10⁻⁴ comparing proprietary with general open-source; Figure 2C). Nonetheless, when considering the loss in accuracy between the baseline performance and performance with distractors, the proprietary models significantly outperformed only the general open-source models (P = .0001; Figure 2D). This suggests that the proprietary models are not only the most robust in the baseline case and with distractors present but also the most resilient to the addition of distractors in the prompt. On the other hand, general open-source models are the least resilient to the addition of distractors.

We confirmed this trend by conducting a Spearman correlation analysis between the baseline performance and change in performance with distractors. Our analysis revealed a moderately strong negative correlation between the baseline accuracy and the loss in accuracy with distractors present (ρ = −0.66, P = .0001; Supplemental Digital Content 3, http://links.lww.com/NEU/F141). In addition, general open-source models clustered around the high accuracy loss and low baseline accuracy while proprietary models clustered around the low accuracy loss and high baseline accuracy. This further highlights the robustness and resilience of proprietary LLMs to added distractors.

Distractor Impact on SANS by Category

After discovering the model classification dependence of the fragility of LLMs to distractors, we analyzed whether the LLMs' performance differed by the section of the SANS examination from which the question came. Without any distractors present, the LLMs performed the most accurately in the Fundamentals, Neuropathology, and Other General sections (P < .0049; Figure 3A). With distractors present, though, Neuropathology was no longer among the most accurately answered sections, and only Fundamentals and Other General remained among the most accurately answered sections (P < .0128; Figure 3B). The same sections—Functional, Spine, Vascular, and Neuroradiology—were consistently the lowest-performing, regardless of the presence of distractors.

FIGURE 3.

Accuracy of LLMs in answering SANS questions with and without distractors present by question section. A, Baseline accuracy of LLMs in predicting answers to SANS questions of each section without any distractors. B, Accuracy of LLMs in predicting answers to SANS questions of each section with distractors present. Different letters indicate significantly different values (P < .05, one-way analysis of variance with Tukey post hoc analysis; n = 28). C, Loss in accuracy of LLMs from the baseline accuracy to the accuracy with distractors present, by model classification and by section. LLMs, large language models; SANS, self-assessment in neurological surgery.

We analyzed the loss in accuracy between the baseline performance and the performance with distractors, both by section and model classification. As expected, for nearly all the sections, the proprietary models exhibited a significantly lower accuracy loss than general open-source models (P < .0330 for all sections except Functional for which P = .0952, Spine for which P = .3139, and Trauma for which P = .4649; Figure 3C).

In addition, we observed that the accuracy loss for Neuropathology was significantly higher than that of other sections only for the general open-source models (P < 10⁻⁴ compared with Functional, P = .0069 compared with Fundamentals, P < 10⁻⁴ compared with Spine, P = .0463 compared with Pain, P < 10⁻⁴ compared with Trauma, P < 10⁻⁴ compared with Vascular for open-source general; P > .3015 for open-source medical; P > .9975 for proprietary; Figure 3C). Thus, the drop in accuracy of LLMs answering Neuropathology questions can primarily be attributed to the higher accuracy loss for Neuropathology questions in general open-source models.

Finally, we notice that the frontier model performance is increasing with time across all model types (Supplemental Digital Content 4, http://links.lww.com/NEU/F142). Among the tested models, none of the medically open-source ones—and only 1 general open-source model—achieved a passing score. A summary of our model statistics in tabular form is presented in Supplemental Digital Content 5, http://links.lww.com/NEU/F143.

DISCUSSION

As LLMs continue to demonstrate remarkable capabilities in medical knowledge recall and board question-answering, interest in their application within neurosurgical training and clinical decision support has surged. From an evaluation of 2904 SANS questions across 28 LLMs, our results confirm this ability of LLMs, both proprietary and open-source, to encode clinical neurosurgical knowledge but also reveal a previously underappreciated vulnerability: When exposed to irrelevant yet clinically familiar distractors, LLMs often exhibit substantial degradation in performance (up to 20.4%), with potentially grave implications for both neurosurgical education and patient care. The safe and effective integration of LLMs into neurosurgical workflows requires a nuanced understanding of their limitations, particularly in high-stakes environments where cognitive precision and distraction filtering are essential.

By systematically embedding noninformative, medically adjacent language into board-style neurosurgical questions, we observed that all tested models—regardless of architecture or training domain—experienced measurable declines in accuracy. This effect was most pronounced among general open-source purpose LLMs, which not only demonstrated the lowest baseline performance but also the greatest susceptibility to distraction (5.6%-20.4% drop in accuracy, depending on the specific model). Proprietary models, while more resilient, were not immune to these effects (for comparison, 1.4%-6.7% drop in accuracy, depending on the specific model). These findings suggest that current LLMs lack the contextual filtering capabilities that are second nature to trained neurosurgeons.

From an educational standpoint, these results are particularly salient. The growing use of LLMs for resident training, simulated board examinations, and just-in-time learning assessments assumes that models can correctly parse clinically relevant content. Our findings are consistent with previous research by Omar et al¹⁵ and Vishwanath et al,¹⁴ who report that even top-performing models can be misled by semantically plausible but irrelevant information. In the context of neurosurgical education, such errors could erode trainee confidence, reinforce misconceptions, or misdirect study focus—particularly when models are trusted to provide reasoning or explanations alongside answers.

Clinically, the distractibility of LLMs raises concerns about patient safety. As LLMs are increasingly embedded into electronic health record systems for ambient documentation, summarization, and even clinical decision support,^19,20 their ability to prioritize relevant information becomes essential. Medical documentation often contains redundant, templated, or copy-forwarded text—elements that a human clinician may intuitively disregard but which could confuse a model. This vulnerability is compounded in high-volume, cognitively intense environments such as the emergency department, trauma bay, or neurosurgical intensive care unit, where clarity and concision are paramount. Such errors, especially when unchecked, have grave implications for clinical decision making and patient safety.

From a technical standpoint, our study underscores the need for robust evaluations that reveal insights about LLMs beyond benchmark accuracy. Recent work has shown that LLMs can fail under subtle perturbations,¹⁷ variation in clinical note type,²⁴ training data-poisoning,²⁵ or even internal tampering,²⁶ and that fine-tuning alone does not confer resistance to distractions.¹⁵ Although medical fine-tuning improves content familiarity, it does not reliably teach models to suppress misleading or nonsalient information—a critical feature for clinical deployment. To this end, future systems may require technical and architectural innovations such as adversarial noise-aware fine-tuning^27,28 and retrieval-augmented prompting restricted to curated templates.^29,30

Importantly, our analysis revealed domain-specific susceptibility: Questions from the Neuropathology section were disproportionately affected by distractors, especially in open-source models. This may reflect the abstract, clinically inconsequent, and often polysemous language used in Neuropathology, which challenges even experienced clinicians. By contrast, sections such as Fundamentals and Other General were more robust, suggesting that the semantic clarity of content plays a key role in LLM resilience.

Limitations

Several factors may temper the generalizability of these findings. First, roughly one-quarter of the original SANS corpus—those items containing images—was excluded. In general, AI systems tend to do worse on the multimodal questions. Alyakin et al¹ evaluated vision-language models such as LLaVA-Med (28.74%), LLaVA-Next (39.81%), CNS-Obsidian (45.25%), Claude-Sonnet-3.5 (56.20%), and GPT-4o (65.60%) on 950 SANS questions that do contain images, a complement of the subset that we used. Combining their evaluation (65.60%) with ours (84.54%), GPT-4o results as a weighted combination yields 79.87%—a result still substantially above the passing range. Second, only a single distractor style was examined, whereas real-world clinical notes may feature longer digressions, copy-and-paste redundancy, and transcription errors. Third, the SANS-CNS data set, while representative of the neurological board examinations, are not completely analogous to the various properties of a real-world clinical environment and tasks required by physicians. Moreover, multiple-choice formats may inadequately assess nuanced clinical reasoning, as LLM performance on such benchmarks can be driven by selection biases and format-specific artifacts, potentially overestimating their practical utility.^31,32 Finally, exact string-matching evaluation cannot award partial credit for conceptually correct but nonidentical answers, potentially underestimating practical competence. Because distractors were artificially infused into a subset of the SANS, caution is warranted in extrapolating to clinical use and should be interpreted as controlled evidence pending evaluation on broader content and real-world clinical materials.

Future work should explore training-time interventions to enhance attention to clinically relevant content, the development of domain-specific safety filters, and real-time monitoring tools to detect LLM uncertainty or confusion. In educational settings, supervised feedback and adversarial testing may help refine model outputs and prevent propagation of errors. As neurosurgery considers the adoption of AI tools, the standard for robustness must match the cognitive rigor of the specialty itself. Prospective evaluations that resemble the real-world clinical workflow should also aim to characterize the effect of distractors and noisy information regarding clinicians.

CONCLUSION

LLMs have reached and, in some cases, surpassed expert-level performance on clean neurosurgical board questions, but that competence is fragile. A single irrelevant sentence containing a medical term can erode accuracy by up to 20.4%, with open-source general-purpose models suffering the most significant declines and proprietary models showing—but not eliminating—residual vulnerability. Because real-world clinical text is replete with extraneous information, robustness to noise is not a peripheral nicety; it is a prerequisite for safe deployment. We therefore advocate that future neurosurgical and broader medical AI benchmarks incorporate explicit noise stress tests and that model developers prioritize training and architectural strategies that confer resilience. Only by demonstrating both high accuracy and high robustness, LLM-based tools can earn the trust required for integration into neurosurgical decision making and, ultimately, improve patient care.

Code availability

Our code is available via request to the corresponding author.

Funding

This study did not receive any funding or financial support.

Disclosures

EKO is supported by the National Cancer Institute's Early Surgeon Scientist Program (3P30CA016087-41S1) and the W.M. Keck Foundation. EKO has equity in Delvi, MarchAI, and Artisight, income from Merck & Co. and Mirati Therapeutics, employment in Eikon Therapeutics, and consulting for Sofinnova Partners and Google. The other authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article.

SUPPLEMENTAL DIGITAL CONTENT

Supplemental Digital Content 1: Figure. Sample SANS question with distract and model response. This question was correctly answered without the distracting sentence (highlighted and bolded) but incorrectly answered with the addition of the distracting sentence.

Supplemental Digital Content 2: Figure. Prompting template for categorizing questions within neurosurgical subtopics.

Supplemental Digital Content 3: Figure. Correlation between baseline accuracy and accuracy loss under distractors present of each LLM by model classification. Each circle represents a different LLM, and points are colored by model classification (orange: general open-source; blue: medical open-source; black: proprietary). The Spearman correlation coefficient is ρ = −.66 with P = .0001.

Supplemental Digital Content 4: Figure. Timeline progression of LLM performance on the neurosurgical board-like examination questions. Each marker represents a different LLM, and points are colored by model classification (orange: general open-source; blue: medical open-source; black: proprietary). Connecting lines are included to show improvement in the baseline performance over time by model classification.

Supplemental Digital Content 5: Table. Summary statistics for model evaluations. P-value is computed through the McNemar test.