Table 1.

Development steps of AS-NTS

Qualitative and quantitative research methodology			Conducted Steps
Development
Development step	1.	Expert groupa	• Literature search • Development of a NTS list and discussion of their relevance for undergraduates
	2.	Focus groupsb and half-structured interviews	• Discussion of NTS • Development of a hierarchical structure of NTS
Validation
Development step	3.	Field Observation	• Testing of feasibility and practicality • Evaluation if the skills were observable
	4.	a) Evaluation questionnaire b) Analyzing the interrater reliability	• Calculation of the content validity indexc • Validation of usability and feasibility • Assessment of NTS in 98 simulation scenarios during emergency training sessions by two/ three independent raters

Legend: ^a The expert group consisted of two anaesthesiology specialists with profound experience in medical education and of two specialists who work on the assessment of psycho-social skills in Multiple Mini Interviews for the purpose of student selection, one of them being a psychologist and one of them highly qualified in medical education. The composition of the expert group was chosen to combine expertise of anaesthesiology, medical education and psychology

^bThe focus group consisted of five anaesthesiology specialists, two male and three female participants, with a median age of 34.8 years. The mother language of all participants is German and they all had completed their residency at the Department of Anaesthesiology, University Medical Center Hamburg Eppendorf. The qualification criteria to join the focus group were 1) to be at anaesthesiology specialist level and 2) to work regularly in anaesthesiology and emergency medicine and in medical teaching. Routine in emergency care was required in order to link the theoretical discussion about the NTS with reality settings

^cThe content validity index for each dimension was calculated, reflecting the proportion of relevance [33]