Table 5.
Non-technical skill and mental workload assessment in surgical robotics. Used abbreviations: RAMIS: Robot-Assisted Minimally Invasive Surgery, OR: Operating Room, VR: Virtual Reality, EEG: electroencephalogram, NASA-TLX: NASA Task Load Index, SURG-TLX: Surgery Task Load Index, NOTSS: Non-Technical Skills for Surgeons, MRQ: Multiple Resources Questionnaire, DSSQ: Dundee Stress State Questionnaire, ECG: electrocardiogram, HR: heart rate, HRV: heart rate variability, RSME: Rating Scale for Mental Effort, PTICSQ: Psychometric Testing of Interpersonal Communication Skills Questionnaire, SAQ: Safety Attitudes Questionnaire, fNIRS: Functional Near-Infrared Spectroscopy, PVT: Psychomotor Vigilance Test, WCST: Wisconsin Card Sorting Test, CITS: Coping Inventory of Task Stress, MSSD: mean square of successive differences between consecutive heartbeats, PEP: time of isovolumetric contraction, HRA: average heart rate, SMEQ: Subjective Mental Effort Questionnaire, LED: Local Experienced Discomfort, SSSQ: Short Stress State Questionnaire, p.: procedures (where no subject data were available), QoE: Quality of Evidence, mod.: moderate.
| Ref. | Date | Subj. | Environment | Input | Measured Feature/NTS | Conclusion | QoE |
|---|---|---|---|---|---|---|---|
| [30] | 2006 | 10 | Dry lab | Skin conductance Self-rating (custom) | Workload Stress | Stress is less in the case of RAMIS compared to traditional MIS. | mod. |
| [58] | 2006 | 5 | VR simulator | NASA-TLX | Workload | Workload can be increased in proportion to delay time with the proposed simulators. | low |
| [31] | 2008 | 15 | Dry lab | DSSQ MRQ CITS | Workload Stress | Stress is less, workload and stress coping strategies are the same in the case of RAMIS compared to traditional MIS. | low |
| [65] | 2009 | 20 | VR simulator | NASA-TLX | Workload | Mimic dV-Trainer shows reasonable workload results. | low |
| [60] | 2009 | 15 | Dry lab | NASA-TLX MRQ | Workload | The usage of the da Vinci 3D view causes less workload compared to the 2D view in some cases. | low |
| [67] | 2009 | 6 | VR simulator | NASA-TLX | Workload | Time delay in teleoperation can significantly increase the workload. | low |
| [85] | 2009 | 16 | Dry lab | MSSD PEP HRA SMEQ LED | Workload Stress | RAMIS causes less cognitive workload compared to traditional MIS. | low |
| [8] | 2010 | 34 | Live porcine | NASA-TLX | Workload | RAMIS poses less mental workload compared to traditional MIS. | mod. |
| [73] | 2010 | 3 | VR simulator | NASA-TLX | Workload | Workload is not improved under delays of 300 ms and 400 ms in the simulated environment. | low |
| [115] | 2010 | 21 | VR simulator | fNIRS | Workload | FNIRS can show the cognitive burden during training. | high |
| [78] | 2012 | 15 | Dry lab | MRQ DSSQ | Workload Stress | Novices have less stress when working with the da Vinci compared to traditional MIS. | low |
| [74] | 2012 | 12 | Dry lab | NASA-TLX | Workload | After the proposed training, mental workload is similar between novices and experts. | low |
| [114] | 2012 | 21 | VR simulator | fNIRS | Cortical activity | There is a significant difference between expert and non-expert subjects with Gaze-Contingent Motor Channeling. | mod. |
| [7] | 2014 | 2 | OR | HR HRV | Stress | RAMIS poses less mental workload compared to traditional MIS. Workload measurement with HRV is cumbersome. | mod. |
| [55] | 2014 | 28 | Dry lab | NASA-TLX | Workload | RAMIS poses significantly better workload perception compared to traditional MIS. | low |
| [54] | 2014 | 13 | Dry lab | NASA-TLX | Workload | Physiological and cognitive ergonomics with robotic surgery are significantly less challenging compared to traditional MIS. | low |
| [53] | 2014 | 52 | VR simulator | NASA-TLX | Workload | Urethrovesical anastomosis VR training improves technical skill acquisition with cognitive demand. | mod. |
| [93] | 2015 | 10 | Dry lab | EEG | Cognitive engagement Mental workload Mental state | Cognitive assessment can define the expertise levels. | high |
| [42] | 2015 | 32 | Dry lab | SURG-TLX RSME Heart rate monitor | Workload HRV | RAMIS poses less mental workload compared to traditional MIS. | mod. |
| [99] | 2015 | 6 | Simulated OR | Expert rating (custom) | Communication Leadership | Repeated simulations and increased leadership mean faster and less flawed conversions in the OR. | mod. |
| [64] | 2015 | 24 | Image display | NASA-TLX | Workload | Increasing the level of cognitive load is significantly increasing the inattention blindness. | mod. |
| [45] | 2015 | 1 | OR | EEG NASA-TLX | Workload Distractions Mental state | Expert surgeons use different mental resources based on their needs. | mod. |
| [98] | 2016 | 89 | OR | Expert rating (custom) | Communication Decision making | RAMIS increases communication requirements for the team of the OR. | mod. |
| [63] | 2016 | 28 | VR simulator | NASA-TLX | Workload | Xperience Team Trainer emphasizes the importance of teamwork. | mod. |
| [81] | 2016 | 32 | OR | PTICSQ SAQ | Communication | There is a significant correlation between team communication and surgical outcome. | mod. |
| [43] | 2016 | 1 | OR | EEG NASA-TLX | Workload | A surgical expert during mentoring concerned while he was observed the surgery. | low |
| [40] | 2016 | 89 | OR | Expert rating (custom) NASA-TLX | Communication Workload | The proposed method is capable of capturing team activities during RAMIS. | mod. |
| [56] | 2016 | 21 | Live porcine VR simulator | NASA-TLX | Workload | Live animal and VR simulator training provide a comparable workload. | low |
| [70] | 2016 | 8 | VR simulator | EEG NASA-TLX | Procedural memory Attention level Workload | EEG can show the learning progress in the case of RAMIS. | high |
| [59] | 2017 | 55 | OR | NASA-TLX | Workload | The study proposes a workload variety analysis with different members of the OR. | mod. |
| [94] | 2017 | 25 p. | OR | NASA-TLX | Workload | NASA-TLX is a useful tool for determining the appropriate staff member mix for RAMIS procedures. | mod. |
| [95] | 2017 | 10 | OR | SURG-TLX | Workload | Mental demands are higher for surgeons at the console than are assisting. | mod. |
| [66] | 2018 | 24 | Live porcine | NASA-TLX | Workload | Single-site access surgery can significantly reduce the workload. | mod. |
| [34] | 2018 | 27 | VR simulator | EEG NASA-TLX | Cognitive features Mental workload Engagement Asymmetry index Brain functional features Communication Integration Recruitment Workload | EEG features can be used for objective non-technical skill assessment. | high |
| [61] | 2018 | 27 | OR | OR efficiency (custom) NASA-TLX | Communication Workload | Anticipation causes shorter operating time. Team familiarity causes less inconveniences. Less anticipation causes less cognitive load. | mod. |
| [62] | 2018 | 32 | VR simulator | NASA-TLX SSSQ MRQ | Workload Stress | Training with a VR simulator can decrease the workload and stress. | mod. |
| [33] | 2018 | 62 | Dry lab Simulated OR | NOTSS | Situational awareness Decision making Leadership Communication Teamwork | Motor imaginary training technique is not effective in non-technical skill training. | mod. |
| [44] | 2018 | 8 | Dry lab | fNIRS SURG-TLX HRV | Prefrontal activation Workload Stress response | RAMIS improves performance during high workload conditions. | high |
| [69] | 2018 | 4 | OR | EEG NASA-TLX | Cognitive features Functional features Mental workload Mental load Engagement Situation awareness Blink rate Asymmetry index Completion time Communication | During a simple surgical task, functional brain features are sufficient to classify mentor–trainee trust. | high |
| [111] | 2018 | 32 | VR simulator | EEG | Electrocortical activity in temporoparietal and left frontal regions | There are significant differences in electrocortical activity between novices and experts. | high |
| [72] | 2018 | 12 | VR simulator | HRV NASA-TLX Wrist motion EMG Electrodermal EEG | Workload Expertise | The proposed skill and workload evaluation framework is accurate. | high |
| [32] | 2019 | 20 | OR | NOTSS NASA-TLX | Situational awareness Decision making Leadership Communication Teamwork Workload | Non-technical skills are associated with team efficiency, surgical flow disruptions and self-perceived performance. | high |
| [75] | 2019 | 5 | OR | NASA-TLX | Workload | Workload is less in the case of robot-assisted submucosal dissection compared to the traditional case. | low |
| [68] | 2019 | 31 | VR simulator | NASA-TLX | Workload | Specific self-directed robotic simulation curriculum was introduced, which can significantly decrease the workload. | mod. |
| [41] | 2019 | 8 | VR simulator | NASA-TLX Eye movements | Workload | Eye movements correlate with the workload. | high |
| [71] | 2019 | 264 p. | OR | NASA-TLX | Workload | Mental workload is similar in the case of RAMIS, traditional MIS, hand-assisted MIS and open surgery. | mod. |
| [57] | 2019 | 30 | Wet lab | NASA-TLX PVT WCST | Workload Concentration Cognitive function | Robotic assistance does not provide less mental workload with novices. Robotic assistance may be mentally taxing for robotic novices. | mod. |
| [76] | 2020 | 7 | OR | NASA-TLX | Workload | RAMIS requires less mental demand and effort compared to open access surgery and traditional MIS. | mod. |
| [46] | 2020 | 26 | Dry lab | Task-evoked pupillary response | Workload | Under high cognitive workload, there can be a divergence in robotic movement profiles between expertise levels. | high |
| [96] | 2020 | n.a. | OR | OTAS NOTSS ICARS NOTECHS II | Situation awareness Decision making Communication Teamwork Leadership Stress | The study proposed a structured approach to the analysis of non-technical skill using extracorporeal videos of both open radical cystectomy and RAMIS radical cystectomy | mod. |