Table 4.
Summary of included studies on gesture and phase recognition.
| Study | Year | Data modality | Data source | Sample size | MLa method | Data availability |
| [78] | 2025 | Video | Microsurgery videos | 132 sutures (11 surgeons) | T-LRCNb; ConvLSTMc | Private |
| [79] | 2025 | Video | Clinical (robotic prostatectomy) | N/Ad | CNNe | Private |
| [80] | 2025 | Video | Clinical (cholecystectomy) | N/A | Action recognition model | Private |
| [81] | 2024 | Video | Clinical (laparoscopic pancreatoduodenectomy) | 69 videos | CNN (TSMf-based) | Private |
| [82] | 2023 | Video | Simulator (low-cost appendectomy trainer) | 43 videos (12 experts, 31 residents) | CNN (ResNet50g); Bi-LSTMh | Private |
| [83] | 2023 | Video; hand pose | Open surgery simulator | 100 videos (25 clinicians) | MSi-TCN++j; I3Dk; YOLOXl | Public |
| [13] | 2022 | Video | JIGSAWSm | 39 videos (8 surgeons) | Calibrated MS-TCN; I3D; SlowFast | Public |
| [84] | 2022 | Video; kinematics | JIGSAWS; RARP-45 | 39 (JIGSAWS); 45 (RARP) | MAn-TCN | Public, private |
| [85] | 2022 | Video | Clinical (laparoscopic TAPP hernia repair) | 119 videos (11 surgeons) | TeCNOo (MS-TCN); HMMp; SV-RCNetq; KNNr | Private |
| [86] | 2021 | Video; kinematics | JIGSAWS; dVRKs simulator (peg transfer) | 75 videos (JIGSAWS); 24 sequences (dVRK) | Relational GCNt (MRG-Netu); ResNet-18; TCN; LSTM | Public, private |
| [5] | 2021 | Video; optical flow | Clinical (robotic prostatectomy) | 511 clips | AlexNet; LSTM; ConvLSTM | Private |
| [28] | 2021 | EEGv | Clinical (robotic prostatectomy) | 34 surgeries (5 surgeons) | Extra Trees; RFw; bagging; KNN | Private |
| [87] | 2020 | Kinematics | JIGSAWS | 39 trials (8 surgeons) | Multitask Bi-LSTM | Public |
| [14] | 2020 | Video | Clinical (laparoscopic colorectal) | 300 videos | CNN (Xception); U-Netx | Private |
| [88] | 2019 | Kinematics | JIGSAWS; V-RASTEDy | 8 (JIGSAWS); 18 (V-RASTED) | Time-delay neural network (TDNNz) | Public |
aML: machine learning.
bT-LRCN: temporal long-term recurrent convolutional network.
cConvLSTM: convolutional long short-term memory.
dN/A: not available.
eCNN: convolutional neural network.
fTSM: temporal shift module.
gResNet50: Residual Network With 50 Layers.
hBi-LSTM: bidirectional long short-term memory.
iMS: multistage.
jTCN: temporal convolutional network.
kI3D: inflated 3D convolutional network.
lYOLOX: You Only Look Once X.
mJIGSAWS: Johns Hopkins University and Intuitive Surgical Inc Gesture and Skill Assessment Working Set.
nMA-TCN: multiaxis temporal convolutional network.
oTeCNO: temporal convolutional network for operating room phase recognition.
pHMM: hidden Markov model.
qSV-RCNet: surgical video recurrent convolutional network.
rKNN: k-nearest neighbors.
sdVRK: da Vinci Research Kit.
tGCN: graph convolutional network.
uMRG-Net: multirelational graph network.
vEEG: electroencephalography.
wRF: random forest.
xU-Net: U-shaped convolutional network.
yV-RASTED: Virtual Reality Assessment of Surgical Tool-Handling and Ergonomic Dexterity.
zTDNN: time-delay neural network.