Protocol for a systematic review of the predictive and translational validity of simulation-based training in neurosurgery

Abstract

Background

Simulation-based training (SBT) has become integral in neurosurgical education, aiming to enhance technical skills and patient safety. While advanced modalities, such as virtual reality (VR), augmented reality (AR), three-dimensional (3D) printing, and artificial intelligence (AI)-enhanced platforms, are increasingly adopted, evidence linking simulator performance to operative competence and patient or system-level outcomes remains limited. This protocol describes a systematic review evaluating the predictive and translational validity of neurosurgical SBT.

Methods

Eligible studies will include neurosurgical trainees and practitioners undergoing SBT or assessment. A comprehensive search strategy, developed with an information specialist, will interrogate MEDLINE, Embase, Scopus, Web of Science, CINAHL, and the Cochrane Library. Inclusion criteria will focus on studies reporting outcomes related to operative competence, technical performance, or patient outcomes; studies assessing only learner satisfaction or knowledge without performance metrics will be excluded. Three reviewers will independently screen studies, extract data using a structured tool, and assess study quality using Messick’s validity framework, the Beckman scale, and the Modified Translational Outcomes of Simulation-Based Mastery Learning (TO-SBML) framework to inform an integrated, structured narrative synthesis.

Discussion

This review will provide an evidence-informed synthesis of the predictive and translational validity of neurosurgical SBT, including AI-enhanced platforms, and identify methodological gaps that limit translation from simulator performance to operative, patient, and system-level outcomes. The findings are expected to inform curriculum design, competency-based assessment, credentialling, and decisions regarding the implementation and resourcing of SBT, while helping prioritise future research in areas where robust translational evidence remains limited.

Systematic review registration

PROSPERO registration: CRD420251131906.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s13643-026-03274-y.

Introduction

Simulation-based training (SBT) is increasingly prominent in neurosurgical education [1], offered through a variety of modalities such as virtual reality (VR) and augmented reality (AR) [2, 3], three-dimensional (3D) printing [4], cadaveric models [5], and artificial intelligence (AI)-enhanced platforms [6]. These technologies aim to improve technical skill acquisition and standardise assessment beyond the operating theatre (OT) [7], addressing limitations of traditional apprenticeship models and aligning with competency-based approaches [8].

Although SBT has been rapidly adopted, robust evidence directly linking it to improved operative performance or patient outcomes is heterogenous and often methodologically limited, as seen in studies evaluating performance in external ventricular drain (EVD) placement [2]. While many platforms have demonstrated varying degrees of validity in controlled settings, few have been rigorously evaluated for their predictive or translational value in real clinical practice using robust, clinically meaningful outcomes. Existing research primarily emphasises feasibility and skill acquisition, with inconsistent, variably reported evidence regarding impact on operative proficiency, error reduction, or clinical outcomes [9]. These fragmented datasets complicate decisions about resource allocation and make it difficult to determine which SBT tools offer the greatest value to education and patient care.

In this context, it is important to distinguish related but conceptually distinct constructs. Validity in SBT concerns the extent to which simulator performance meaningfully represents or predicts performance in real operative settings [10], whereas educational effectiveness reflects whether SBT leads to measurable improvements in learners’ skills, behaviours, or progression [11], and clinical impact refers to downstream effects on patient-level outcomes such as complications, morbidity, or mortality [12]. This review focuses primarily on the predictive and translational validity of SBT in neurosurgery, that is, the extent to which SBT performance and assessment relate to subsequent operative competence and, where available, clinical or system-level outcomes [13, 14]. In practice, predictive and translational validity lie on a continuum from SBT performance through intraoperative behaviour to patient-level and system-level outcomes; in this review, we distinguish them primarily for organisational purposes while acknowledging areas of overlap. This conceptual continuum is operationalised in the Methods section through a pre-specified rule-set for classifying studies according to their predominant outcome type and highest translational level. Given the largely observational nature of the anticipated evidence base, translational validity will be interpreted as an observed association between SBT and clinical or system-level outcomes rather than as direct causal inference, with methodological limitations including confounding and selection bias explicitly considered during synthesis. Within contemporary medical education theory, validity is conceptualised as a unified construct supported by multiple sources of evidence, as described in Messick’s framework [15]. Accordingly, this review applies the complementary frameworks of Messick’s validity framework, the Beckman scale [16], and the modified translational outcomes (TO) of simulation-based mastery learning (SBML) model to characterise assessment validity [14], the quality of competence evaluation, and the extent to which observed effects translate beyond SBT performance to behavioural, patient, and system-level outcomes.

In neurosurgery, the relationship between fidelity and predictive or translational validity is also procedure dependent. For relatively circumscribed tasks such as EVD placement, the key performance requirements can often be captured with a focused high-physical-fidelity head model that allows skin incision, burr hole creation, and freehand catheter insertion into the ventricle. In this context, predictive validity is comparatively easier to evaluate because the skill set is narrow, the procedural endpoints are well defined, and operative performance can be quantified using objective metrics such as first-pass success or tip position on imaging. By contrast, complex operations such as retrosigmoid craniotomy and microvascular decompression of the trigeminal nerve involve multiple interdependent steps, higher cognitive load, and greater patient risk, and are typically undertaken by more senior trainees. Establishing predictive validity for such procedures requires simulators that approximate not only local anatomy but also multi-step exposure, microsurgical dissection, and complication management, and it is more challenging to attribute downstream clinical outcomes to specific simulated behaviours. Accordingly, in this review, we interpret predictive and translational findings in light of both simulator fidelity and procedural complexity, rather than assuming that validity is equally testable across all neurosurgical tasks.

Against this backdrop, the drive towards competency-based training [17] and workplace-based assessments (WBA) [18] has added urgency to questions of authenticity and relevance, particularly in highly specialised fields such as neurosurgery, where operative exposure is becoming more limited and tolerance for error is minimal. Neurosurgical procedures often have shallow learning curves [19] and should not, where possible, be ‘learnt on patients’ alone, making the specialty especially dependent on high-fidelity SBT to support both skill acquisition and assessment. Syntheses that aggregate all surgical specialties may obscure these neurosurgery-specific demands, as patterns driven by higher-volume, lower-risk procedures can dominate and effectively swallow the signal from neurosurgical contexts. Here, there remains scepticism about whether skills demonstrated in SBT reliably transfer to complex, real-world patient care [20].

Previous neurosurgical SBT reviews, including the two-part synthesis by Patel et al. [9, 21] and the meta-analysis by Meling [22], have provided valuable overviews of available platforms, validity evidence, and skill-acquisition outcomes, with some patient-level findings. However, they have not systematically examined the predictive relationship between simulator performance and subsequent intraoperative behaviour, nor explicitly structured findings within translational frameworks that distinguish internal validity from real-world clinical impact. This protocol extends that work by explicitly mapping predictive validity (the extent to which SBT performance predicts operative performance) and translational validity (the extent to which SBT is associated with measurable clinical or patient-level outcomes), incorporating AI-enhanced, data-driven SBT platforms, and applying a stricter focus on performance outcomes and patient-centred outcomes to structure appraisal and synthesis (Fig. 1). For SBT to be successfully integrated into neurosurgical curricula, more than technological effectiveness is needed. There must be clear evidence for optimal use cases, strong institutional and faculty engagement, appropriate investment of resources, and a commitment to treating SBT as a core component of surgical education.

Fig. 1.

Conceptual continuum from simulator performance through intraoperative behaviour to patient and system outcomes, illustrating how predictive and translational validity, and corresponding TO-SBML levels, are mapped across simulation, clinical, and health system environments

Main research questions

1. What is the evidence that performance in neurosurgical SBT predicts subsequent operative performance and clinical competence (predictive validity)?

2. What is the evidence that neurosurgical SBT is associated with improvements in patient-level and system-level outcomes (translational validity)?

Secondary objectives

1. Map domains where SBT shows educational or clinical.

2. Identify gaps by design, modality, outcomes, etc.

3. Develop evidence-informed recommendations for curriculum, assessment, and future research.

Methods

This protocol has been registered in the International Prospective Register of Systematic Reviews (PROSPERO) (Registration ID: CRD420251131906). It is reported in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 guidelines [23]. Eligibility criteria were defined using the PICO framework to include studies evaluating neurosurgical trainees undergoing SBT, compared to conventional training or no intervention, with outcomes related to operative competence, clinical performance, or patient safety:

• Population

  • ◦ Participants in included studies will be restricted to neurosurgical doctors (trainees, fellows) and practising neurosurgeons, including consultants, who have undergone SBT and/or assessment. Studies including mixed learner cohorts (e.g. neurosurgical and non-neurosurgical trainees) will be eligible only where neurosurgical data are reported separately or where neurosurgeons constitute a clearly defined subgroup for whom outcomes can be disaggregated. Studies in which neurosurgical data cannot be isolated will be excluded. Studies involving participants outside of formal neurosurgical training pathways, such as medical students or physician associates (PAs), will also be excluded. This restriction reflects the focus on predictive and translational validity of SBT for neurosurgical practice, recognising that early feasibility or face validity work in junior learners is less likely to provide directly interpretable evidence on operative transfer for neurosurgeons.
• Intervention

  • ◦ This review will consider a wide range of SBT modalities, including high-fidelity technologies such as VR and AR, physical simulators such as 3D-printed anatomical models, cadaveric dissection-based approaches, and hybrid models that combine physical and digital components.
  • ◦ All studies that use these modalities for technical skills training, rehearsal, or assessment (irrespective of the physical or virtual nature of the simulation) will be considered for inclusion. Studies evaluating non-technical skills (NTS) training alone (e.g. decision-making, teamwork) without technical performance or clinical outcomes will be excluded, as NTS SBT is typically evaluated using different theoretical frameworks and outcome measures. Their transfer to operative or patient-level outcomes is unlikely to be captured by the predictive and translational validity focus of this review.
  • ◦ The scope further extends to studies evaluating AI-enhanced SBT platforms, whether used for training, automated and adaptive feedback, assessment, or real-time performance analytics.
  • ◦ AI-enhanced SBT and platforms incorporating automated or adaptive performance analytics will be explicitly flagged during data extraction. Where study numbers permit, we will conduct a pre-specified subgroup analysis comparing AI-enhanced versus non-AI-enhanced SBT platforms within each SBT modality and procedural domain in the narrative synthesis.
• Comparison

  • ◦ This review will include studies that evaluate neurosurgical SBT interventions, regardless of whether a direct, non-SBT comparator arm is present. Eligible study designs include single-arm studies (e.g. pre/post or longitudinal observational designs), multi-arm trials (e.g. comparing SBT with conventional training), and mixed-methods studies where SBT is the primary focus of the intervention.
  • ◦ Single-arm studies without non-SBT comparators will be included to map the breadth of available evidence and to identify patterns across SBT modalities, procedural domains, and trainee levels. However, they will not be used to support conclusions regarding the comparative effectiveness of SBT. Stronger inferences regarding predictive and translational validity will be drawn preferentially from comparative study designs (e.g. randomised controlled trials and controlled cohort studies). Findings from single-arm studies will be interpreted as hypothesis-generating and presented descriptively during synthesis.
  • ◦ The primary aim is to map and synthesise the breadth of evidence on the predictive and translational value of SBT in neurosurgical education and clinical practice. Studies that do not involve a SBT intervention will be excluded.
• Outcome

  • ◦ Eligible studies must report outcomes that reflect the predictive or translational value of SBT. These include (i) operative competence assessed using validated tools (e.g. WBAs [18], Objective Structured Assessment of Technical Skills (OSATS) [24]); (ii) technical performance metrics such as operative time, error or complication rates, and procedural success; and/or (iii) patient-centred outcomes including morbidity, mortality, and length of hospital stay. Studies reporting only learner satisfaction, confidence, or theoretical knowledge without performance-based or clinical outcomes will be excluded.
  • ◦ We will also include objectively quantified but non-validated performance metrics (e.g. bespoke checklists, motion-tracking, time-and-error measures), with their psychometric properties assessed explicitly in the validity appraisal phase. These simulator-derived metrics will generally be interpreted as evidence of internal skill acquisition or early predictive validity where they demonstrate meaningful associations with independent measures of surgical experience or competence. However, in the absence of demonstrated associations with independent operative performance or patient-level outcomes, they will not be considered evidence of operative transfer or definitive translational validity.
  • ◦ For AI-enhanced SBT, we will additionally extract AI-specific performance outputs where reported (e.g. automated error classifications, composite proficiency scores, adaptive difficulty levels, or progression decisions). When these AI-derived metrics are compared with independent measures of operative performance, they will be treated as evidence of predictive validity, and when they are linked to behavioural, patient-level, or system-level outcomes, they will contribute to translational validity. AI-derived performance metrics will be interpreted cautiously because they may be influenced by algorithmic opacity, overfitting to simulator-specific datasets, limited external validation, and the potential for circularity where AI platforms both generate and evaluated performance. Unless supported by independent operative performance measures or behavioural, patient-level, or system-level outcomes, AI-derived outputs will be regarded as surrogate markers of SBT performance rather than independent evidence of predictive or translational validity, and these considerations will inform the interpretation and synthesis of evidence of AI-enhanced SBT studies.
  • ◦ Classification rule-set for predictive and translational validity. Each included study will be classified according to the highest level(s) of evidence it provides along the simulator-to-clinical-impact continuum:

    • ▪ Predictive validity—evidence linking simulator-derived performance (e.g. automated metrics, proficiency scores) to an independent measure of operative performance (e.g. WBA, structured intraoperative rating, blinded video review).
    • ▪ Translational validity—evidence linking SBT to behavioural change in clinical practice (e.g. reduced errors, faster operative times) or to patient-level or system-level outcomes (e.g. complication rates, length of stay, economic impact).

Where a single study contributes evidence at more than one level, its findings will be reported within each relevant category rather than forced into a single classification. Throughout this review, both forms of validity are interpreted as observed associations; causal inference will not be claimed unless the study design explicitly supports it.

Eligibility criteria—types of studies

This review will include randomised controlled trials (RCTs), cohort studies, case-control studies, and observational primary studies that report relevant performance or clinical outcomes. Systematic reviews, meta-analyses, narrative reviews, opinion pieces, editorials, and conference abstracts lacking complete data will be excluded as units of analysis; existing reviews may be used only to identify additional eligible primary studies. This approach ensures that only primary research with sufficient methodological detail is included in the synthesis.

Search strategy, pilot testing, and information sources

A comprehensive search strategy will be developed iteratively by one reviewer (A.R.) in collaboration with an information specialist (K.W.). Core concepts will be identified and expanded using both medical subject headings (MeSH) and free-text keywords, adapted to each database. AI-related terminology was not used as a primary search concept because the review focuses on SBT platforms rather than AI-enhanced platforms more broadly. In the SBT literature, AI-enabled training environments are typically described using SBT-related terminology such as simulation, VR, or AR. Including AI as a standalone search concept substantially increased retrieval of unrelated clinical AI applications outside the SBT domain. Therefore, SBT terms were prioritised to maintain search specificity.

The search will be conducted across MEDLINE (via Ovid), PubMed, Cochrane Library, Embase, CINAHL, Web of Science, and Scopus, with no date restrictions but limited to English-language publications due to resource and translation constraints. Grey literature sources such as theses, dissertations, and technical reports will not be included because these sources may lack sufficient methodological detail and have not undergone formal peer review, which may limit the ability to evaluate validity evidence. Excluding grey literature may increase the risk of publication bias, particularly if studies reporting positive findings from SBT are more likely to undergo peer review and publication than studies with neutral or negative findings. This potential source of bias will be considered when interpreting the review findings. Reference lists of included studies and relevant systematic reviews will also be screened for additional eligible records. Final database-specific strategies will be provided in Appendix 1, in accordance with PRISMA-P guidance to facilitate transparency and reproducibility.

Study selection process

All search results will be imported into Covidence™, where duplicates will be automatically removed. Title and abstract screening will be conducted in duplicate. Three reviewers (A.R., D.S., and L.D.) will independently screen all records from a shared screening pool. Covidence’s allocation system will ensure that each record receives two independent votes. Reviewer contribution and progress will be monitored using Covidence’s Team Settings and Team Progress functions to ensure balanced workload and complete coverage of the dataset. Any disagreements at either stage will be resolved through discussion or consultation with a third reviewer (A.D.). Reasons for exclusion at the full-text stage will be documented. If multiple publications arise from the same underlying study population, they will be identified and treated as a single study to avoid double counting, with the most comprehensive report serving as the primary source. The study selection process will be reported in accordance with PRISMA guidelines (Supplementary Material 1) and illustrated using a PRISMA flow diagram.

Data collection process and data items

Data will be extracted using a custom-designed extraction tool developed in Microsoft Excel™ (Supplementary Material 2), which captures key information related to study characteristics, population details, simulation modalities, outcomes, and validity evidence. Two reviewers will independently extract data in duplicate from all included studies using this template, with discrepancies resolved through discussion or, if necessary, adjudication by a third reviewer (A.D.). The extraction form will be piloted on a subset of studies to ensure consistency and clarity before full data extraction commences. Data will be extracted using a structured template developed specifically for this review (see Table 1 and Supplementary Material 2).

Table 1.

Summary of data items to be extracted from included studies

Category	Variables to be extracted	Extracted as
Study characteristics	Author(s), year, country, study design, sample size	Text
Population	Trainee level (e.g. resident, fellow, consultant); inclusion/exclusion criteria	Text/categorical
Simulation intervention	Modality (virtual reality (VR), augmented reality (AR), cadaveric, hybrid); fidelity; purpose (training, assessment, both); artificial intelligence (AI) use	Categorical/text
Comparator(s)	Description of comparator group (conventional, alternate simulation, or none)	Text/categorical
Outcomes measured	Operative competence (Objective Structured Assessment of Technical Skills (OSATS), workplace-based assessments (WBA)); technical performance (errors, operative time); patient outcomes (morbidity, mortality, length of stay)	Numeric, rating scale, text (as reported)
Validity evidence	Messick’s domains: content, response process, internal structure, relations to other variables, consequences	Numeric score (0–3), absent (0), weak (1), moderate (2), strong (3)
Translational impact	Level on Modified Translational Outcomes of Simulation-Based Mastery Learning (TO-SBML) framework	Ordinal (levels 1–5)
Rater-based evaluation	Beckman rating scale	Numeric score (0–2), narrative comment

All screening reviewers will complete data extraction for all included studies. Missing or unclear information will be flagged and clarified via author contact where feasible. For each study, we will extract the time interval between completion of SBT and any subsequent operative or WBA, the duration of follow-up for patient-level or system-level outcomes, and whether skill retention or decay is evaluated (e.g. through repeated simulator or clinical evaluations over time). These timing variables will be reported descriptively and considered when interpreting the strength and durability of any observed predictive or translational effects.

Risk of bias and quality assessment

The quality and validity of all included studies will be evaluated using a structured, reproducible system modelled on the methodology adopted from previous literature [9, 21]. This combines three complementary frameworks:

• oMessick’s validity framework [15]: Each study will be evaluated across five domains: content, response process, internal structure, relations to other variables (including predictive validity and any AI-derived performance metrics), and consequences. Predictive validity will be captured primarily within the ‘relations to other variables’ domain, where simulator performance is compared with independent measures of operative performance.
• oBeckman et al. rating scale [16]: This scale will be applied to rate the clarity, structure, and reliability of rater-based assessments of clinical competence within each included study.
• oTO-SBML [14]: Each study will be categorised by its highest achieved level of translational effectiveness, from internal skill acquisition to observed impact on patient outcomes. TO-SBML levels will operationalise translational validity by classifying the extent to which SBT interventions translate into behavioural change and patient-level or system-level outcomes.

These frameworks were selected in preference to generic training evaluation hierarchies (e.g. Kirkpatrick model [25]) because they offer the psychometric and translational granularity required to appraise validity evidence in procedural simulation contexts and have been applied in comparable surgical simulation reviews.

To ensure consistent appraisal, we have developed a custom study-level tool (Supplementary Material 3) that presents the domains and ratings from the three frameworks in a single structured table to facilitate transparent comparison across studies. Messick’s validity domains and the Beckman rating scale will be assessed using a four-point scale (absent = 0, weak = 1, moderate = 2, strong = 3), and the completed appraisal table will be provided as a supplementary file. The highest TO-SBML level achieved by each study will be recorded according to the framework’s established categories of translational impact.

For each study, we will report domain-specific Messick ratings, Beckman scores, and the highest TO-SBML level rather than deriving a single composite score. These ratings will be used descriptively to characterise the strength and type of validity evidence across studies and will not be used as thresholds for study inclusion or exclusion. In particular, TO-SBML levels will be used descriptively to generate a validity ‘profile’ per study, to map how many studies reach higher translational levels (including patient-outcome levels), and to structure subgrouping within the narrative synthesis (e.g. contrasting studies limited to internal skill acquisition with those demonstrating patient-level or system-level effects). Where differences arise between validity evidence and translational outcomes (e.g. strong internal validity evidence but limited downstream clinical outcomes), these patterns will be reported descriptively and discussed in the narrative synthesis. Inter-rater agreement for the appraisal process will be evaluated on a subset of studies using Cohen’s kappa statistic approach, with discrepancies resolved through discussion or consultation with a third reviewer (A.D.) where necessary. During synthesis, we will qualitatively explore whether studies with stronger validity profiles (e.g. higher ‘relations to other variables’ ratings or higher TO-SBML levels) tend to report more robust predictive or translational outcomes, but we will not apply numeric cut-offs for inclusion, weight effect estimates, or conduct formal meta-regression based on these scores.

Integrative rationale

These frameworks are applied in combination because they capture complementary dimensions of validity. Messick’s framework evaluates the validity of simulation-based assessments, the Beckman scale assesses the quality of rater-based evaluation methods, and the modified TO-SBML framework classifies the extent to which observed effects translate from simulator performance to behavioural, patient-level, or system-level outcomes. Together, they enable an integrated assessment of whether simulation-based assessments are psychometrically robust, how reliably competence is evaluated, and whether these measures demonstrate meaningful translational impact. We anticipate that many studies will demonstrate satisfactory internal validity evidence but fewer will provide strong evidence relating simulator performance to independent operative performance or achieve higher TO-SBML levels reflecting behavioural or patient-level outcomes. For the purposes of this review, a strong validity profile will be characterised by consistent evidence across multiple Messick domains, acceptable Beckman ratings where rater-based assessments are used, and evidence extending beyond simulator performance towards predictive and translational validity.

Data management and missing data

Study screening and data extraction will be managed using Covidence systematic review software. Extracted data will be stored securely in a version-controlled Excel™ file within a secure Microsoft SharePoint™ folder. In cases of missing or unclear data, study authors will be contacted for clarification; if clarification is not obtained, data gaps will be documented and addressed as limitations within the review narrative.

Data synthesis

Given anticipated heterogeneity in study designs, SBT modalities, outcome measures, and participant populations, quantitative pooling is unlikely to be appropriate. A structured narrative synthesis will therefore be undertaken. Quantitative meta-analysis is not planned, and the review will be primarily descriptive and hypothesis-generating. If, contrary to expectation, we identify clusters of studies with closely comparable interventions, populations, and outcome measures and consistently reported effect estimates, we will consider a limited meta-analysis within those clusters. However, we anticipate that substantial heterogeneity in simulation modalities, procedural domains, trainee levels, comparators, and outcome constructs will make routine effect size pooling misleading, because it would obscure construct differences and combine incompatible endpoints (e.g. technical performance scores and patient-level outcomes) into single summary estimates.

We will treat predictive and translational validity as points along a continuum from simulator performance to patient-level and system-level outcomes and, where studies contribute to multiple stages along this pathway, we will report them accordingly rather than forcing rigid categorisation into a single level. Outcomes will be grouped into conceptually similar categories (e.g. operative performance scores, error counts, time-and-motion metrics, complication rates, patient-centred outcomes) and the direction of effect will be summarised qualitatively within these categories rather than statistically pooled. Where different instruments measure similar constructs, we will compare effect directions and highlight differences in measurement tools, scoring methods, and follow-up intervals to aid interpretation, with particular attention to outcomes that most directly reflect predictive and translational validity. The synthesis will be reported with reference to the SWiM (Synthesis Without Meta-analysis) reporting guideline [26], where appropriate.

The narrative synthesis will be organised by SBT modality, procedural domain, trainee level, and translational outcome level. Procedural heterogeneity will be handled by grouping operations into broader technical domains, (e.g. cranial, spinal, neurovascular/microsurgical, and neuroendoscopic), commenting on procedural complexity and risk profile, and exploring whether patterns of predictive or translational validity differ across domains and complexity levels. Within these groupings, we will qualitatively grade the strength of evidence (e.g. stronger, limited, inconclusive) based on methodological rigour, consistency of findings, and directness of outcomes, and, where data allow, we will stratify results by procedure complexity and trainee seniority. When findings conflict across SBT modalities or study designs, we will explore methodological explanations such as differences in study design quality and risk of bias, trainee experience level, simulator modality and fidelity, exposure duration and intensity, and outcome measurement tools (including timing of assessment), giving greater interpretive weight to studies with lower risk of bias and higher translational outcome levels. Consistent with this interpretive approach, conclusions regarding the effectiveness of SBT will be based primarily on comparative study designs, whereas evidence from single-arm studies will be synthesised descriptively to characterise the breadth and distribution of the available literature.

Within each combination of SBT modality, procedure, and trainee level, AI-enhanced simulations (including platforms with automated or adaptive performance analytics) will be identified explicitly so that, where numbers permit, their predictive and translational patterns can be summarised as a distinct subgroup. In addition to mapping the existing evidence, we will use the patterns identified across SBT modality, procedural domain, trainee level, and translational outcome level to develop pragmatic, evidence-informed recommendations for integrating SBT into neurosurgical training and assessment, highlighting procedures or tasks where SBT appears to offer the greatest gains in technical completeness, efficiency, or patient-relevant outcomes. Potential meta-biases, including publication bias and selective outcome reporting, will be considered qualitatively during the synthesis and interpretation of findings.

Studies will be organised and synthesised according to:

• Thematic groupings:

  • ◦ Simulation modality (e.g. VR, AR, cadaveric, hybrid; AI-enhanced versus non-AI-enhanced platforms within these categories)
  • ◦ Procedural domain (e.g. cranial, spinal, neurovascular or microsurgical, neuroendoscopic, and image-guided or navigation-based procedures, with exemplars such as aneurysm clipping, EVD placement within these categories)
  • ◦ Trainee level (e.g. trainee, fellow, consultant)

• Key outputs:

  Summary tables and visual figures to present:

  • ◦ Study characteristics
  • ◦ Validity evidence and methodological appraisal (reported using Messick domain ratings, Beckman scores, and TO-SBML translational levels)
  • ◦ Translational outcomes and clinical relevance

• Narrative themes explored:

  Summary tables and visual figures to present:

  • ◦ Patterns in predictive and translational validity
  • ◦ Strength of supporting evidence
  • ◦ Gaps in the current literature and areas for future investigation

Amendments and timeline

Any amendments to this protocol will be documented with the date and rationale, and updated accordingly on the PROSPERO registration record. Data screening and extraction commenced following protocol registration and are currently ongoing. The systematic review is expected to be completed following completion of study selection, data extraction, and synthesis.

Discussion

This protocol outlines the methodology for a systematic review of the predictive and translational validity of SBT in neurosurgery, including AI-enhanced and data-driven platforms. Although evidence linking SBT performance to operative competence and patient outcomes remains heterogenous, some studies have reported improvements in clinical performance and, in some cases, patient-level outcomes [9, 21, 22]. This review will apply validated appraisal frameworks and structured narrative synthesis to clarify the extent to which these findings support predictive and translational validity beyond simulator-based skill acquisition.

The anticipated contribution of this review is twofold. First, it will provide an evidence-informed overview of how SBT modalities, including AI-enhanced platforms, are currently being evaluated within neurosurgical training and how they may translate into real-world operative and clinical outcomes. Second, it will highlight common methodological gaps, including heterogeneity in comparators, limited reporting of patient-level outcomes, and inconsistent use of validated metrics, thereby guiding priorities for future research and curriculum development.

Study selection and data extraction will be performed independently in duplicate, using a piloted extraction template, and study quality will be assessed through rigorous, reproducible frameworks (Messick’s validity framework [15], the Beckman scale [16], and the modified TO-SBML classification [14]). Together with a structured narrative synthesis, this approach is intended to provide a transparent and nuanced overview of how neurosurgical SBT, including AI-enhanced platforms, contributes to predictive and translational outcomes within contemporary training pathways.

This review will provide an evidence-informed synthesis to guide neurosurgical educators, policymakers, healthcare organisations, and training bodies in decisions about curriculum design, competency-based assessment, credentialing [27, 28], and investment in SBT. At present, SBT is increasingly recognised internationally as a core component of high-quality healthcare education and training, with recent consensus statements advocating its integration throughout healthcare curricula, assessment, credentialing, and clinical practice [29]. However, within neurosurgery, there remains limited understanding of the extent to which improvements demonstrated in SBT environments translate into operative performance, patient outcomes, and wider system-level benefits. Without a clearer synthesis of the available literature, educational practice and investment decisions are likely to continue along this trajectory, with widespread adoption of SBT despite persistent uncertainty regarding its translational impact. By identifying where robust predictive and translational validity has been demonstrated, and where important evidence gaps remain, this review will help prioritise future research, including methodologically rigorous studies evaluating operative competence, patient outcomes, and the downstream clinical and economic impact of SBT. Ultimately, this work will strengthen the basis for informed decisions regarding the implementation, funding, and resourcing of SBT within neurosurgical training, with the aim of improving surgical performance, patient safety, and healthcare resource utilisation.

Strengths and limitations

This protocol incorporates several methodological strengths, including a comprehensive, librarian-supported search strategy; the use of validated frameworks for quality and translational assessment; and independent, duplicate review at all stages. The proposed systematic evaluation of the predictive and translational impact of SBT in neurosurgery is intended to address a recognised gap in the current evidence base. Anticipated limitations include the exclusion of non-English literature, which may introduce geographic and innovation bias by under-representing relevant simulation and AI-enhanced SBT research published in non-English-language journals, potential heterogeneity among included studies, and the likelihood that the available evidence will be dominated by lower-level study designs and surrogate outcomes, with limited availability of high-level endpoints such as patient safety metrics or long-term clinical impact. These challenges are compounded by structural features of neurosurgical SBT research, including typically small sample sizes, variation in institutional training environments, limited standardisation of technical performance metrics across simulators, and the inherent difficulty of isolating the effect of SBT on patient-level outcomes from other confounding educational and service factors. Additionally, variability in SBT modalities, outcome measures, and reporting quality may restrict direct quantitative comparison between studies and limit the feasibility of formal meta-analysis. Nevertheless, this heterogeneity is expected to provide a comprehensive overview of the breadth and diversity of the current evidence base. The review may also be affected by publication bias and selective outcome reporting, as studies demonstrating positive educational or clinical effects may be more likely to be published or to report favourable outcomes. We deliberately restricted inclusion to technical SBT, recognising that NTS SBT (e.g. teamwork, communication) represent an essential but conceptually distinct literature that warrants separate evaluation using different validity and outcome frameworks. These limitations will be transparently acknowledged when interpreting the findings. Any patient-level or long-term clinical outcomes identified will be interpreted cautiously, and conclusions will reflect the methodological quality and observational nature of the available evidence. Accordingly, this review is intended to characterise patterns and gaps in the existing literature rather than to provide definitive estimates of the effectiveness of SBT on operative competence or patient outcomes. Importantly, an absence of high-level evidence should not be interpreted as evidence that SBT is ineffective, but may instead reflect the practical and methodological challenges of evaluating translational outcomes in neurosurgical education.

Authors’ contributions

A.F.R. conceived the study, designed the protocol, and drafted the manuscript. A.J.D. provided expert input on study design and critical revision of the manuscript. K.W. developed and refined the search strategy in collaboration with A.F.R. Title and abstract screening were performed by A.F.R., D.S., and L.D. C.M.C. provided senior supervision and oversight of the study. All authors (A.F.R., A.J.D., D.O.K., D.C., D.S., L.D., K.W., G.Z., V.H., A.P., A.A., and C.M.C.) contributed to the development of the protocol, provided critical feedback, and approved the final manuscript. A.F.R. is the guarantor of the review.

Funding

No funding was required to carry out this protocol. A.F.R. is supported by an AO Medical Education Fellowship for the conduct of the subsequent systematic review; however, this fellowship had no role in the design, preparation, or writing of the present protocol.

Data availability

All data generated or analysed during this study, including the extracted datasets and study appraisal scores, will be available from corresponding author upon reasonable request.

Declarations

Ethics approval and consent to participate

This review will use only aggregated data reported in published studies; no identifiable patient-level data will be directly accessed or re-analysed, and therefore separate ethical approval is not required.

Competing interests

The author team declare no conflicts of interest.