Table 3.

Behavioral rating systems in traditional surgery compared to ICARS, the only established non-technical skill assessment metric particularly for RAMIS [21,22].

	Revised NOTECHS	NOTSS	OTAS	ICARS
Date	2008	2006	2006	2017
Reference	[100]	[101]	[102]	[22]
Non-technical skills	Communication and interaction Situational awareness Team skills Leadership and management Decision making	Situational awareness Decision making Task management Leadership Communication Teamwork	Task checklist Shared monitoring Communication Cooperation Coordination Shared leadership	Communication and teamwork Leadership Decision making Situational awareness Cope with stress and distractors
Content validity		✓	✓	✓
Construct validity			✓	✓
Inter-rater reliability	✓	✓	✓	✓
Sensitivity	n.a.	not acceptable in some categories	n.a.	n.a.
Feasibility	✓ (especially for self-assessment)	✓	limited to certain procedures	✓