Reporting guidelines for health care simulation research: Extensions to the CONSORT and STROBE statements

Introduction

Simulation has seen growing use in health care as a ‘tool, device and/or environment (that) mimics an aspect of clinical care’¹ in order to improve health care provider performance, health care processes and, ultimately, patient outcomes.^1–5 The use of simulation in health care has been accompanied by an expanding body of simulation-based research (SBR) addressing educational and clinical issues.^6–15 Broadly speaking, SBR can be broken down into two categories: (1) research addressing the efficacy of simulation as a training methodology (ie, simulation-based education as the subject of research); and (2) research using simulation as an investigative methodology (ie, simulation as the environment for research).^{16 17} Many features of SBR overlap with traditional clinical or educational research. However, the use of simulation in research introduces a unique set of features that must be considered when designing the methodology, and reported when publishing the study.^16–19

As has been shown in other fields of medicine,²⁰ the quality of reporting in health professions education research is inconsistent and sometimes poor.^{1 11 21–23} Systematic reviews in medical education have quantitatively documented missing elements in the abstracts and main texts of published reports, with particular deficits in the reporting of study design, definitions of independent and dependent variables, and study limitations.^21–23 In research specific to simulation for health care professions education, a systematic review noted many studies failing to ‘clearly describe the context, instructional design or outcomes’.¹ Another study found that only 3% of studies incorporating debriefing in simulation education reported all the essential characteristics of debriefing.¹¹ Failure to adequately describe the key elements of a research study impairs the efforts of editors, reviewers and readers to critically appraise strengths and weaknesses^{24 25} or apply and replicate findings.²⁶ As such, incomplete reporting represents a limiting factor in the advancement of the field of simulation in health care.

Recognition of this problem in clinical research has led to the development of a growing number of reporting guidelines in medicine and other fields, including the Consolidated Standards of Reporting Trials (CONSORT) statement for randomised trials,^27–30 the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement for observational studies^{31 32} and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement,^33–35 among more than 250 others.³⁶ Transparent reporting of research allows readers to clearly identify and understand ‘what was planned, what was done, what was found, and what conclusions were drawn’.³¹ In addition to these statements, experts have encouraged³⁷ and published extensions to existing statements that focus on specific methodological approaches^{38 39} or clinical fields.^{40 41} In this study, we aimed to develop reporting guidelines for SBR by creating extensions to the CONSORT and STROBE statements specific to the use of simulation in health care research. These reporting guidelines are meant to be used by authors submitting manuscripts involving SBR, and to assist editors and journal reviewers when assessing the suitability of simulation-based studies for publication.

Methods

The study protocol was reviewed by the Yale University Biomedical Institutional Review Board and was granted exempt status. We conducted a multistep consensus process based on previously described steps for developing health research reporting guidelines.⁴² These steps involved (1) developing a steering committee; (2) defining the scope of the reporting guidelines; (3) identifying a consensus panel; (4) generating a list of items for discussion; (5) conducting a consensus meeting; and (6) drafting reporting guidelines and an explanation and elaboration document.

Development of the steering committee

A steering committee was formed consisting of 12 members with expertise in simulation-based education and research, medical education research, study design, statistics, epidemiology and clinical medicine. The steering committee defined the scope of the reporting guidelines, identified participants for the consensus process, generated a premeeting survey, planned and conducted the consensus meeting and, ultimately, drafted and refined the final version of the reporting guidelines and the explanation and elaboration document.

Defining the scope of the reporting guidelines

To clarify the scope of the reporting guideline extensions, we defined simulation as encompassing a diverse range of products including computer-based virtual reality simulators, high-fidelity and static manikins, plastic models and task trainers, live animals, inert animal products, human cadavers and standardised or simulated patients (ie, individuals trained to portray a patient). Our definition excluded research using computational simulation and mathematical modelling, as the guidelines were developed for research using human participants, either as learners or health care providers.¹ The steering committee determined to create reporting guidelines encompassing two categories of SBR: (1) studies evaluating simulation for educational use; and (2) studies using simulation as investigative methodology.¹⁶ We identified the CONSORT²⁸ and STROBE^{31 32} statements as reflecting the current reporting standards in health care research and aimed to develop extensions of these two statements for quantitative SBR. The CONSORT statement and extensions were developed for randomised trials, and the STROBE statement and extensions were developed for observational studies (cohort, case-control and cross-sectional study designs). Our guideline extensions are not intended for qualitative research, mixed-methods research or for validation studies.

Identification of consensus panel participants

The steering committee aimed to identify a consensus group with a broad range of expertise in SBR, including experience in conducting single and multicentre simulation-based studies, expertise in educational research, statistics, clinical epidemiology and research methodology, and with varying clinical backgrounds. We invited the editor-in-chief and editorial board members of three health care simulation journals: Simulation in Healthcare, BMJ Simulation & Technology Enhanced Learning and Clinical Simulation in Nursing, and editorial board members from two medical education journals: Medical Education and Advances in Health Sciences Education. In total, 60 expert participants were invited to complete the online survey.

Generating a list of items for discussion

Prior to the consensus meeting, we surveyed the expert participants via a premeeting survey (http://www.surveymonkey.com) to identify items in the CONSORT and STROBE statements that required an extension for SBR. The survey included all items from the CONSORT and STROBE statements and was pilot tested among steering committee members before being posted online. Participants were asked to provide suggested wording for the items they identified as requiring an extension. Participants were also given the option of suggesting new simulation-specific items for the CONSORT and STROBE statements. On the basis of methods previously used to develop extensions to the CONSORT statement,⁴⁰ we used a cut-off of endorsement by at least one-third of respondents to identify high-priority items for discussion during the consensus meeting.

Consensus meeting

A 5 h consensus conference was conducted in January 2015 in New Orleans, USA, during the annual International Network for Simulation-based Pediatric Innovation, Research and Education (INSPIRE) meeting. The initial 60 consensus panel participants were invited to attend the consensus conference as well as INSPIRE network members (ie, clinicians, researchers, educators, psychologists, statisticians and epidemiologists). The INSPIRE network is the world's largest health care simulation research network with a proven track record of conducting rigorous simulation-based studies in health care.^43–50

The results of the online survey were circulated to each member of the steering committee, who were then assigned to review specific items from the CONSORT and STROBE statements based on their expertise. The consensus meeting started with a brief didactic presentation reviewing the CONSORT and STROBE statements, followed by a description of the study objectives and consensus process. In small groups, each steering committee member led a discussion with four or five individuals tasked with determining if a simulation-specific extension was required for their assigned items, and if so, to recommend wording for the extension. Consensus panel participants were evenly distributed among small groups and specifically assigned to review items based on their area of expertise. High-priority items were discussed at length, but all other checklist items were also discussed in the small groups.

Following small group discussion, the recommended simulation-specific extensions for the CONSORT and STROBE statements were presented to the entire group of participants. Each proposed extension was discussed before recommended wording was established. Minutes from the small and large group discussions were used to inform the development of the explanation and elaboration document.⁴²

Drafting reporting guidelines

The proposed extensions were circulated for comment among all meeting participants and consensus panel participants who could not attend the meeting. The steering committee used the comments to further refine the extension items. To evaluate these items in practice, four members of the steering committee independently pilot tested the CONSORT and STROBE statements with simulation-specific extensions. They used two published SBR studies (ie, one for each type of SBR), while ensuring one study was a randomised trial and the other an observational study. Feedback from pilot testing informed further revisions. The final reporting guidelines with extensions were circulated to the steering committee one last time to ensure the final product accurately represented discussion during and after the consensus conference. An explanation and elaboration document was developed by the steering committee to provide further detail for each item requiring a simulation-specific extension.⁴²

Results

Premeeting survey

There was a 75% response rate for the survey, with 45 of the 60 participants completing the entire survey. An additional 12 (20%) other participants partially completed the survey. Of the 57 participants who responded to the survey, 17 were medical journal editors or editorial board members, 24 had advanced degrees (master's degree, PhD), 16 with advanced degrees in medical education or educational psychology, 6 were nurses, 1 was a psychologist and 54 were physicians (representing anaesthesiology, critical care, emergency medicine, paediatrics and surgery). Of the three participants who did not complete the survey, two were physicians and one was a scientist. The results of the survey are described in online supplementary Digital Content (see supplementary table, Digital Content 1, Survey Responses).

Consensus meeting

In total, 35 consensus panel participants who completed the premeeting survey attended the consensus conference. An additional 30 attendees were INSPIRE network members. Of the 65 total attendees at the consensus conference, 12 were medical journal editors or editorial board members, 18 had advanced degrees (master's degree, PhD), 4 were nurses, 1 was a psychologist and 60 were physicians (representing anaesthesiology, critical care, emergency medicine, paediatrics and surgery).

Eleven simulation-specific extensions were recommended for the CONSORT statement: item 1 (title and abstract), item 2 (background), item 5 (interventions), item 6 (outcomes), item 11 (blinding), item 12 (statistical methods), item 15 (baseline data), item 17 (outcomes and estimation), item 20 (limitations), item 21 (generalisability) and item 25 (funding). Participants agreed upon the importance of describing the rationale for and design of the simulation-based intervention. As many simulation-based studies use assessment tools as an outcome measure, participants thought it was important to report the unit of analysis and evidence supporting the validity and reliability of the assessment tool(s) when available. In the Discussion section, participants thought it was important to describe the limitations of SBR and the generalisability of the simulation-based outcomes to clinical outcomes (when applicable). Participants also agreed it was important to identify the simulator brand used in the study and if conflict of interest for intellectual property existed among investigators. The group did not feel that modifications to the CONSORT flow diagram were required for SBR. See table 1 for CONSORT extensions for SBR.

Table 1.

Simulation-based research extensions for the CONSORT statement

Item	Item no	CONSORT description (randomised controlled trials)	Extension for simulation-based research
Title and abstract	1a, 1b	1a: Identification as a randomised trial in the title 1b: Structured summary of trial design, methods, results and conclusions	In abstract or key terms, the MeSH or searchable keyword term must have the word ‘simulation’ or ‘simulated’.
Introduction
Background	2a, 2b	2a: Scientific background and explanation of rationale 2b: Specific objectives or hypotheses	Clarify whether simulation is subject of research or investigational method for research.
Methods
Trial design	3a, 3b	3a: Description of trial design (such as parallel, factorial) including allocation ratio 3b: Important changes to methods after trial commencement (such as eligibility criteria), with reasons
Participants	4a, 4b	4a: Eligibility criteria for participants 4b: Settings and locations where the data were collected
Interventions	5	The interventions for each group with sufficient details to allow for replication, including how and when they were actually administered	Describe the theoretical and/or conceptual rationale for the design of each intervention. Clearly describe all simulation-specific exposures, potential confounders and effect modifiers.
Outcomes	6a, 6b	6a: Completely defined prespecified primary and secondary outcome measures, including how and when they were assessed 6b: Any changes to trial outcomes after the trial started, with reasons	In describing the details of methods of assessment, include (when applicable) the setting, instrument, simulator type, timing in relation to the intervention, along with any methods used to enhance the quality of measurements. Provide evidence to support the validity and reliability of assessment tools in this context (if available).
Sample size/study size	7a, 7b	7a: How sample size was determined 7b: When applicable, explanation of any interim analyses and stopping guidelines
Randomisation: sequence generation	8a, 8b	8a: Method used to generate the random allocation sequence 8b: Type of randomisation; details of any restriction (such as blocking and block size)
Randomisation: allocation concealment mechanism	9	Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned
Randomisation: implementation	10	Who generated the random allocation sequence, who enrolled participants and who assigned participants to interventions
Blinding (masking)	11a, 11b	11a: If done, who was blinded after assignments to interventions (eg, participants, care providers, those assessing outcomes) and how 11b: If relevant, description of the similarity of interventions	Describe strategies to decrease risk of bias, when blinding is not possible.
Statistical methods	12a, 12b	12a: Statistical methods used to compare groups for primary and secondary outcomes 12b: Methods for additional analyses, such as subgroup analyses and adjusted analyses	Clearly indicate the unit of analysis (eg, individual, team, system) and identify repeated measures on subjects, and describe how these issues were addressed.
Results
Participant flow (a diagram is strongly recommended)	13a, 13b	13a: For each group, the numbers of participants who were randomly assigned, received intended treatment and were analysed for the primary outcome 13b: For each group, losses and exclusions after randomisation, together with reasons
Recruitment	14a, 14b	14a: Dates defining the periods of recruitment and follow-up 14b: Why the trial ended or was stopped
Baseline data	15	A table showing baseline demographic and clinical characteristics of each group	In describing characteristics of study participants, include their prior experience with simulation and other relevant features as related to the intervention(s).
Numbers analysed	16	For each group, number of participants (denominator) included in each analysis and whether analysis was by original assigned groups
Outcomes and estimation	17a, 17b	17a: For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% CI) 17b: For binary outcomes, presentation of absolute and relative effect sizes is recommended	For assessments involving more than one rater, inter-rater reliability should be reported.
Ancillary analyses	18	Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing prespecified from exploratory
Adverse events	19	All important harms or unintended effects in each group (for specific guidance, see CONSORT for harms)
Discussion
Limitations	20	Trial limitations, addressing sources of potential bias, imprecision and, if relevant, multiplicity of analyses	Specifically discuss the limitations of simulation-based research.
Generalisability	21	Generalisability (external validity) of the trial findings	Describe the generalisability of simulation-based outcomes to patient-based outcomes (if applicable).
Interpretation	22	Interpretation consistent with results, balancing benefits and harms and considering other relevant evidence
Other information
Registration	23	Registration number and name of trial registry
Protocol	24	Where the full trial protocol can be accessed, if available
Funding	25	Sources of funding and other support (such as supply of drugs), role of funders	List simulator brand and if conflict of interest for intellectual property exists.

CONSORT, Consolidated Standards of Reporting Trials; MeSH, medical subject heading.

Ten extensions were drafted for the STROBE statement: item 1 (title and abstract), item 2 (background/rationale), item 7 (variables), item 8 (data sources/measurement), item 12 (statistical methods), item 14 (descriptive data), item 16 (main results), item 19 (limitations), item 21 (generalisability) and item 22 (funding). A similar emphasis was placed on the importance of describing all simulation-specific exposures, confounders and effect modifiers, as was discussed for the CONSORT. Other extensions for the STROBE were under similar categories as the proposed extensions for the CONSORT. See table 2 for STROBE extensions for SBR.

Table 2.

Simulation-based research extensions for the STROBE statement

Item	Item no	STROBE description (observational studies)	Extension for simulation-based research
Title and abstract	1a, 1b	1a: Indicate the study's design with a commonly used term in the title or the abstract. 1b: Provide in the abstract an informative and balanced summary of what was done and what was found.	In abstract or key terms, the MeSH or searchable keyword term must have the word ‘simulation’ or ‘simulated’.
Introduction
Background/rationale	2	Explain the scientific background and rationale for the investigation being reported.	Clarify whether simulation is subject of research or investigational method for research.
Objectives	3	State specific objectives, including any prespecified hypotheses.
Methods
Study design	4	Present key elements of study design early in the paper.
Setting	5	Describe the setting, locations and relevant dates, including periods of recruitment, exposure, follow-up and data collection.
Participants	6a, 6b, 6c	6a: Cohort study: give the eligibility criteria, and the sources and methods of selection of participants. Describe methods of follow-up. Case-control study: give the eligibility criteria, and the sources and methods of case ascertainment and control selection. Give the rationale for the choice of cases and controls. Cross-sectional study: give the eligibility criteria, and the sources and methods of selection of participants. 6b: Cohort study: for matched studies, give matching criteria and number of exposed and unexposed. 6c: Case-control study: for matched studies, give matching criteria and the number of controls per case.
Variables	7	Clearly define all outcomes, exposures, predictors, potential confounders and effect modifiers. Give diagnostic criteria, if applicable.	Describe the theoretical and/or conceptual rationale for the design of the intervention/exposure. Describe the intervention/exposure with sufficient detail to permit replication. Clearly describe all simulation-specific exposures, potential confounders and effect modifiers.
Data sources/measurement	8	For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group.	In describing the details of methods of assessment, include (when applicable) the setting, instrument, simulator type, timing in relation to the intervention, along with any methods used to enhance the quality of measurements. Provide evidence to support the validity and reliability of assessment tools in this context (if available).
Bias	9	Describe any efforts to address potential sources of bias.
Study size	10	Explain how the study size was arrived at.
Quantitative variables	11	Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen, and why.
Statistical methods	12a, 12b, 12c, 12d, 12e	12a: Describe all statistical methods, including those used to control for confounding. 12b: Describe any methods used to examine subgroups and interactions. 12c: Explain how missing data were addressed. 12d: Cohort study: if applicable, explain how loss to follow-up was addressed. Case-control study: if applicable, explain how matching of cases and controls was addressed. Cross-sectional study: if applicable, describe analytical methods taking account of sampling strategy. 12e: Describe any sensitivity analyses.	Clearly indicate the unit of analysis (eg, individual, team, system) and identify repeated measures on subjects, and describe how these issues were addressed.
Results
Participants	13a, 13b, 13c	13a: Report the numbers of individuals at each stage of the study—eg, numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up and analysed. 13b: Give reasons for non-participation at each stage. 13c: Consider use of a flow diagram.
Descriptive data	14a, 14b, 14c	14a: Give characteristics of study participants (eg, demographic, clinical, social) and information on exposures and potential confounders. 14b: Indicate the number of participants with missing data for each variable of interest. 14c: Cohort study: summarise follow-up time—eg, average and total amount.	In describing characteristics of study participants, include their prior experience with simulation and other relevant features as related to the intervention(s).
Outcome data	15	Cohort study: report numbers of outcome events or summary measures over time. Case-control study: report numbers in each exposure category or summary measures of exposure. Cross-sectional study: report numbers of outcome events or summary measures.
Main results	16a, 16b, 16c	16a: Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% CIs). Make clear which confounders were adjusted for and why they were included. 16b: Report category boundaries when continuous variables were categorised. 16c: If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period.	For assessments involving more than one rater, inter-rater reliability should be reported.
Other analyses	17	Report other analyses done—eg, analyses of subgroups and interactions and sensitivity analyses.
Discussion
Key results	18	Summarise key results with reference to study objectives.
Limitations	19	Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss direction and magnitude of any potential bias.	Specifically discuss the limitations of simulation-based research.
Interpretation	20	Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies and other relevant evidence.
Generalisability	21	Discuss the generalisability (external validity) of the study results.	Describe the generalisability of simulation-based outcomes to patient-based outcomes (if applicable).
Other information
Funding	22	Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based.	List simulator brand and if conflict of interest for intellectual property exists.

MeSH, medical subject heading; STROBE, STrengthening the Reporting of OBservational studies in Epidemiology.

For the CONSORT and STROBE statements, extensive discussion occurred in the consensus meeting related to the educational intervention and controlling for simulation-specific variables that pose as potential threats to the internal validity of simulation studies. A group of consensus panel participants with expertise in simulation-based education and instructional design used their knowledge of educational theory, existing educational research guidelines⁵¹ and systematic reviews of SBR^{1 5–8 11} to address this issue (table 3). Table 3 offers an additional checklist of key elements specific to SBR, for item 5 (interventions) on the CONSORT statement and item 7 (variables) on the STROBE statement, that should be reported for all simulation studies, for both the intervention and control groups (if applicable).

Table 3.

Key elements to report for simulation-based research

Elements*	Subelements†	Descriptor
Participant orientation	Orientation to the simulator	Describe how participants were oriented to the simulator (eg, method, content, duration).
	Orientation to the environment	Describe how participants were oriented to the environment (eg, method, content, duration).
Simulator type16	Simulator make and model	Describe the simulator make and model.
	Simulator functionality	Describe functionality and/or technical specifications that are relevant to the research question. Describe modifications, if any. Describe limitations of the simulator.
Simulation environment16	Location	Describe where the simulation was conducted (eg, in situ clinical environment, simulation centre, etc)
	Equipment	Describe the nature of the equipment available (eg, type, amount, location, size, etc)
	External stimuli	Describe any external stimuli (eg, background noise)
Simulation event/scenario16	Event description	Describe if the event was programmed and/or scripted (eg, orientation to event, scenario progression, triggers). If a scenario was used, the scenario script should be provided as an appendix.
	Learning objectives	List the learning objectives and describe how they were incorporated into the event
	Group vs individual practice	Describe if the simulation was conducted in groups or as individuals.
	Use of adjuncts	Describe if adjuncts (eg, moulage, media, props) were used.
	Facilitator/operator characteristics	Describe experience (eg, clinical, educational), training (eg, fellowship, courses) and profession.
	Pilot testing	Describe if pilot testing was conducted (eg, number, duration, frequency).
	Actors/confederates/standardised/simulated patients16	Describe experience (eg, clinical, educational), training (eg, fellowship, courses), profession and gender. Describe various roles, including training, scripting, orientation, and compliance with roles.
Instructional design (for educational interventions)19 or exposure (for simulation as investigative methodology)16	Duration	Describe the duration of the educational intervention. If the intervention involves more than one segment, describe the duration of each segment.
	Timing	Describe the timing of the educational intervention relative to the time when assessment/data collection occurs (eg, just-in-time training).
	Frequency/repetitions	Describe how many repetitions were permitted and/or the frequency of training (eg, deliberate practice).
	Clinical variation	Describe the variation in clinical context (eg, multiple different patient scenarios).
	Standards/assessment	Describe predefined standards for participant performance (eg, mastery learning) and how these standards were established.
	Adaptability of intervention	Describe how the training was responsive to individual learner needs (eg, individualised learning).
	Range of difficulty	Describe the variation in difficulty or complexity of the task.
	Non-simulation interventions and adjuncts	Describe all other non-simulation interventions (eg, lecture, small group discussion) or educational adjuncts (eg, educational video), how they were used, and when they were used relative to the simulation intervention.
	Integration	Describe how the intervention was integrated into curriculum.
Feedback and/or debriefing11	Source	Describe the source of feedback (eg, computer, simulator, facilitator).
	Duration	Describe the amount of time spent.
	Facilitator presence Facilitator characteristics	Describe if a facilitator was present (yes/no), and if so, how many facilitators. Describe experience (eg, clinical, educational), training (eg, fellowship, courses), profession and gender.
	Content	Describe content (eg, teamwork, clinical, technical skills and/or inclusion of quantitative data).
	Structure/method	Describe the method of debriefing/feedback and debriefing framework used (ie, phases).
	Timing	Describe when the feedback and/or debriefing was conducted relative to the simulation event (eg, terminal vs concurrent).
	Video	Describe if video was used (yes/no), and how it was used.
	Scripting	Describe if a script was used (yes/no) and provide script details as an appendix.

*These elements may apply to the simulation intervention (eg, randomised control trial (RCT) or observational study with simulation as an educational intervention) or when simulation is the environment for research (eg, RCT or observational study using simulation as an investigative methodology). Elements should be described in sufficient detail to permit replication.

†Description required only if applicable.

In modelling the explanation and elaboration document after other similar documents published in conjunction with reporting guidelines,^{28 32} we provide a specific example for each item requiring a new extension coupled with the background and rationale for including that information for that item. We encourage readers to refer to the explanation and elaboration document to seek further detail about the nature and type of recommended reporting for each new extension (see text, online supplementary Digital Content 2, Explanation and Elaboration of the Simulation-Specific Extensions for the CONSORT and STROBE Statements).

Discussion

We have developed reporting guidelines for SBR by creating extensions to the CONSORT²⁸ and STROBE³¹statements. These new extensions were developed via a consensus building process with multiple iterative steps involving an international group of experts with diverse backgrounds and expertise. By creating extensions to the CONSORT and STROBE statements that can be applied to studies in both categories of SBR, we have developed reporting guidelines that are applicable to the majority of studies involving simulation in health care research. To further assist authors in reporting SBR studies, we have published an explanation and elaboration document as an appendix that provides specific examples and details for all the new simulation-specific extensions for the CONSORT and STROBE statements.

The CONSORT and STROBE statements with accompanying SBR extensions are meant to serve as a guide to reporting. As with other CONSORT and STROBE statements, the items are not meant to ‘prescribe the reporting… in a rigid format’, but rather the ‘order and format for presenting information depends on author preferences, journal style, and the traditions of the research field’.^{28 31} We encourage authors to refer to the explanation and elaboration document that provides details regarding specific elements related to individual items that should be reported for SBR. The use of reporting guidelines can have positive effects on various health care simulation stakeholders, including funders of SBR and those applying for funding (ie, use as a template for grant applications), educators (ie, use as a training tool) and students (ie, use to develop protocols for coursework or research).³³ The application of these reporting guidelines will help to enhance the quality of reporting for quantitative SBR and assist journal reviewers and editors when faced with assessing the strengths and weaknesses of simulation-based studies in health care.^{24 52 53} We encourage journals publishing SBR to consider endorsing the simulation-specific extensions for the CONSORT and STROBE statements and adding these to their ‘Instructions for Authors’.

SBR has several unique factors that prompted us to develop simulation-specific extensions for the CONSORT and STROBE statements. First, there are a wide variety of simulators and simulation modalities available for use in research.¹⁶ This, coupled with a plethora of instructional design features in simulation-based educational research, makes describing the simulation intervention a critically important component of any educational study involving simulation (table 3).^{6 8 19} Second, SBR provides opportunity for the investigator to standardise the simulated environment and/or simulated patient condition. Standardisation of the environment and patient condition allows the investigator to account for many of the potential threats to internal validity that are associated with simulation. Clear reporting of standardisation strategies helps the reader understand how the independent variable was isolated (table 3).¹⁶ Third, many simulation studies involve capturing outcomes from a variety of data sources (eg, observation, video review, simulator data capture). When assessment instruments are used (eg, expert raters assessing performance), it is imperative to discuss the psychometric properties of these instruments.⁵ Existing guidelines fall short in this regard, and these new guidelines help to address this issue. Last, simulation-based studies assessing outcomes in the simulated environment only (eg, clinical performance) should attempt to provide evidence to support how the findings in the simulated environment translate to a valid representation of performance in the real clinical environment.³ By doing so, authors help to convey the relevance and importance of their findings.

Limitations

Our consensus process has several limitations. Although we had a 75% response rate for our survey, an additional 20% of participants only partially completed the survey. This may have potentially introduced a selection bias, although the survey represented only one step in our consensus building process. We include a wide variety of experts in our consensus meeting, but many of them had a paediatric clinical background. We minimised this potential bias by ensuring that each breakout group had at least one expert participant with a background outside of paediatrics. Furthermore, the principles of SBR are common across specialties and professions, and INSPIRE network members represent researchers who are recognised internationally for being leaders in SBR. We based our reporting guidelines on the CONSORT and STROBE guidelines developed by clinical researchers. Other guidelines could have been used as a starting point such as the American Educational Research Association (AERA) standards developed in 2006.⁵⁴ Our logic was to start with reporting guidelines that were applicable to all types of research, thus providing us more flexibility in generating extensions for both types of SBR. Cross-checking against the AERA guideline does not reveal areas we might have missed. While we tried to develop reporting guidelines for all types of SBR, we recognise there may be specific types of research that may require new items or different extensions. For example, studies designed to evaluate the validity of simulation-based assessments vary in their reporting requirements. The STAndards for Reporting of Diagnostic accuracy (STARD) statement⁵⁵ addresses these points, and a recent review operationalised these standards and applied them to SBR.⁵⁶ Other reporting guidelines that might be amenable for simulation-specific extensions include the COnsolidated criteria for REporting Qualitative research (COREQ),⁵⁷ and the Standards for QUality Improvement Reporting Excellence (SQUIRE)⁵⁸ guidelines for reporting quality improvement studies. As the field of SBR grows, the simulation-specific extensions for the CONSORT and STROBE statements may need to be revised or refined. We encourage authors, reviewers and editors to visit our website (http://inspiresim.com/simreporting/) and provide feedback that will be used to inform subsequent revisions to these reporting guidelines.

Conclusions

The unique features of SBR highlight the importance of clear and concise reporting that helps readers understand how simulation was used in the research. Poor and inconsistent reporting makes it difficult for readers to interpret results and replicate interventions, and hence less likely for research to inform change that will positively influence patient outcomes. The use of standardised reporting guidelines will serve as a guide for authors wishing to submit manuscripts for publication, and in doing so, draw attention to the important elements of SBR and ultimately improve the quality of simulation studies conducted in the future.